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ABSTRACT 


Once the military helicopter pilot deploys aboard a 
naval vessel he leaves behind all training platforms, 
short of the actual aircraft, that present enough fidelity 
for him to maintain the highest levels of readiness. To 
that end, this thesis takes a preliminary step in creating 
a trainer that places the pilot in an immersive and 
familiar environment to exercise myriad piloting tasks as 
faithfully and as rigorously as in actual flight. 

The focus of this thesis it to assess the viability 
of a chromakeyed augmented virtual environment (ChrAVE) 
trainer embedded into a helicopter for use in maintaining 
certain perishable skills. Specifically this thesis will 
address the task of helicopter low-level land navigation. 

The ChrAVE was developed to substantiate the 
viability of having embedded trainers in helicopters. The 
ChrAVE is comprised of commercial off the shelf (COTS) 
equipment on a transportable cart. 

In determining whether a system such as the ChrAVE is 
viable as a laboratory for continued training in virtual 
environment, the opinion of actual pilots that were tasked 
with realistic workloads was used. Additionally, 
empirical data was collected and evaluated according to 
the subject pool's thresholds for acceptable low-level 
navigation performance. 
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I. INTRODUCTION 

A. PROBLEM STATEMENT 

Classically, the design of military helicopter 
simulators has not been user-centered with regard to the 
embarkation requirements of military pilots. Simply put, 
simulators that replicate helicopter cockpits are land 
based and not viable for deployment due in part to their 
large footprint. While such simulators meet the spectrum 
of needs of military pilots in garrison, they are rendered 
useless to pilots once they deploy. In this vein, they are 
machine-centered designs. 

Personal computers, while highly deployable, have also 
fallen short in meeting the needs of the pilot because the 
interfaces between human and computer are not realistic; 
they too are not designed around the user. Personal 
computer (PC) applications remove the pilot user from his 
normal environmental interfaces (i.e. the cockpit) and 
require additional learning on the part of the pilot in 
order to use the application. For example in the real 
world a pilot (one of two in the aircraft) may affect a 
turn to the right by moving the flight controls himself or 
issuing a command to the other pilot in the aircraft. 
However, while using a PC based trainer the same turn made 
using a mouse or keyboard. This is far less intuitive to 
the trained pilot. In fact, one could argue that even a 
joystick, which presents a stronger metaphor to the pilot, 
is just as bad or even worse than a mouse. Granted, some 
features of PC trainers are not intended to replicate real 
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world experience, but those that are, should do so with 
high fidelity and accuracy. 

Once the military helicopter pilot deploys aboard a 
naval vessel en route to destine areas of flight, he leaves 
behind all training platforms, short of the actual 
aircraft, that present enough fidelity to the pilot for him 
to maintain the highest levels of training. To that end, 
this thesis takes a preliminary step in creating a trainer 
that places the pilot in an immersive and familiar 
environment to exercise myriad piloting tasks as faithfully 
and as rigorously as in actual flight. 

B. MOTIVATION 

Classically as helicopter pilots prepare for a six- 
month worldwide deployment aboard ship they spend the six 
months prior to the deployment honing myriad helicopter 
piloting and aircrew coordination skills. The process 
creates a plateau of heightened skills for the entire 
squadron by the day of departure. However, while en route 
to destine areas of flight, skills atrophy due to reduced 
amounts of flight time and the flight regimes available. 
Adding to skill erosion is the non-availability of shore- 
based flight simulators while at sea. 

Navigation is the chief requisite skill for many 
aggregate piloting tasks and is therefore a sound choice to 
begin study of performance in visual flight simulators. 
The skill of helicopter overland navigation is difficult, 
if not impossible to maintain while at sea. Visual flight 
simulators for helicopter pilots are not available aboard 
ship due in part to space constraints. Additionally, 
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suitable overland low-level visual 


do 


helicopter trainers 

not meet the needs of the helicopter community. 

Current visual systems are poor at rendering an 
appropriate image to the helicopter pilot. Flight at high 
altitude is sufficiently portrayed but the earth assumes a 
distant 2-dimensional posture. It is at low altitude where 
helicopter pilots navigate, take cover, and mask their 
exposure to enemy observation and fires. Maintaining these 
profiles while navigating, specifically associating 3-D 
terrain images with a 2-D map does not appear possible in 
present visual systems. 

At low-levels visual cues such as optic flow, motion 
parallax, interposition, etc. have a tremendous impact upon 
how an environment (real or virtual) is perceived. 
Accordingly, the head and eye movements of the pilot, which 
are a form of interaction with the environment, provide 
vital feedback. In the real world head and eye movements 
dictate the points of view while the mind produces the 
visual cue over time (i.e. the cue of motion parallax 
requires motion over a period of time for the mind to 
establish or recognize the cue) . In a virtual environment 
(VE) , the correct points of view can only be rendered if 
the system generating the VE accounts for the pilot's head 
position and orientation. Current simulators do not track 
the movements of the pilot's head (figure 1); current 
simulators render views that are dynamic for the moving 
aircraft but static to the movements of the pilot within 
that aircraft. Current simulators provide a static point 
of view for each region or area of view they display. 
Pilots expecting to 'interact' visually with such systems 
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are denied the information they seek; although head and eye 
movements may happen on a subconscious level, the mind is 
still at work trying to produce visual cues over time. 
Frustration, anxiety or even cybersickness can result. 
Often the pilot aborts all attempts to visually interact 
with the environment; head movements cease and the pilot 
assumes a television viewing posture. This is a form of 
negative training. 
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Moreover, while pilots are en route to destine areas 
of flight their resources for interaction are basically 
limited to conventional 2-D 1:50,000 scale maps. While 

this may aid in associating relative locations to one 
another, the ability to visualize being 'in' the terrain is 
drastically limited. It is this visualization that can aid 
in associating world to map. 

It is assumed that in trying to develop a training 
system for military helicopter pilots, it is important to 
embrace the entire spectrum of user needs. A system that 
is easily deployable and presents a cockpit of high 
fidelity for intuitive use would be most attractive. 
Logically, a trainer that is embedded into the actual 
aircraft would likely meet these requirements. Such a 
training system would take advantage of the actual cockpit, 
utilize the actual instruments (cyclic and collective) of 
the human-machine interface as the interface between human 
and computer, and be as deployable as the actual aircraft 
with minimal additional equipment footprint. 

However, before jumping into full production of 
embedded trainers, we must first explore the viability of 
such a trainer. Such exploration requires three basic 
steps; (1) research into the psychology and potential of 
training via simulators, (2) the production of a fully 
operational embedded trainer, and (3) verification of the 
results of using such a trainer. This thesis is part of 
step one. A prototype has been developed to explore 
initial potential of an embedded trainer. If this research 
proves viable, successive steps may follow. 
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Such an undertaking is vast when considering the 
multitude of tasks required to fly a military helicopter in 
all its possible profiles. For this preliminary research 
the fundamental task of low-level terrain navigation shall 
be explored. Navigation is a fundamental underlying 
function to most every task of helicopter aviation. 

C. THESIS OVERVIEW 

If embedded trainers are ever to come to fruition, 
they must first be explored as viable and practicable with 
regard, first, to user performance (user-centered design), 
and second, to machine implementation (machine-centered 
design). In the aforementioned three-step research 
process, this thesis is part of the first step; 
implementing a system and running a preliminary experiment. 

Present landlocked motion simulators have near-full 
fidelity of the cockpit. Dimensions of a simulator's 
cockpit are identical to actual aircraft. Simulator 
instrument displays provide flight information that is 
indiscernible from actual aircraft. Flight control 
response and feedback, while very good, still have room for 
improvement. Without doubt, simulators can make the 
greatest gains by improving the interactive graphics of the 
virtual environment. User-centered designs must embrace 
the natural way in which a pilot interacts with the 
environment by creating motion parallax with dynamic head 
movements. 

The best of simulators poorly emulate the feedback 
required for developing or solely maintaining the skillful 
dexterity required to manipulate the flight controls of an 
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actual helicopter. This research shall not try to 
duplicate such simulators. However, this research shall 
attempt to focus on the task of low-level navigation, which 
by its very nature requires no skillful dexterity of the 
flight controls by the navigating pilot when the duties of 
aircrew are properly divided. Clearly, if replicating the 
task of low-level navigation is not viable, then more 
complex tasks such as faithfully emulating high fidelity in 
flight control feedback for the maintenance of skilled 
dexterity will not be viable. 



-Controls position craft in VE space 
-Head-tracker positions view from aircraft (primary 
point of view + head-tracked ddta) 

•Mxer receives: 

-blue-screen and coclqjjt video from camera 
-VE video from CPU 
-HMD presents augmented reality 


Head-mounted Display 
(HMD) 


A Basic Chromakey Augmented Virtual 
Environment (ChrAVE) Implementation 


Camera 


Head-tracker 


CPU 


Mixer 


Figure 2. Basic ChrAVE Implementation. 


As such, a prototype platform has been devised from 
which research into the psychology and potential types of 
training can be launched. This system adopts a generic 
cockpit environment visually and tactilely while augmenting 
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it with an inside-out visual representation of virtual 
terrain. It employs chromakey technology in order to mix 
the views of the real and virtual worlds, i.e. the user can 
see himself, his cockpit, and virtual terrain fused 
together. 

If this prototype's development progresses properly, 
an actual embedded system could assume a small permanent 
footprint in existing or future aircraft. This system 
could potentially be used by both ship and land-based 
deployed helicopter forces. The ChrAVE was developed to 
substantiate the viability of having embedded trainers in 
helicopters. The ChrAVE is comprised of commercial off the 
shelf (COTS) equipment on a transportable cart. 

D. RESEARCH QUESTIONS 

The basic question of this research is whether or not 
an embedded trainer is viable in theory, early design, and 
preliminary experimentation. This system will be 
considered viable if it is practical to produce, use, and 
is effective as a training device. These parameters can be 
difficult to ascertain so the following criteria are 
offered as determining viability: 

1) Opined by experts (qualified pilots) as having 
value and being practicable. Certainly qualified pilots 
have valuable opinions about their training needs. This 
research shall attempt to address their needs as completely 
as possible. If the collective expert opinion is that such 
an embedded trainer is not valuable or practicable after 
use in its present state then it should be redesigned or 
abandoned. Moreover, since qualified pilots are the end- 



user, they can state the likelihood of their willingness to 
use such a training device, a factor that may prove to be 
the most valuable indicator that this research is 
developing favorably. If their collective willingness is 
low, the system should be redesigned or abandoned. 

2) Proven by empirical data as having value and being 
practicable. If the collective performance of experts 
having used such an embedded trainer (prototype or full 
production model) is unsatisfactory, then an embedded 
trainer is not valuable or practicable in its present state 
and should be redesigned or abandoned. 

3) Acceptable cost of both prototype and full 
production model. If in pursuit of the user's needs, cost 
outweighs the benefits of having such a system, it shall 
not be considered viable in its current state and should be 
redesigned or abandoned. 

4) Vacuously viable. In the absence of finding 
evidence that proves embedded trainers non-viable, they 
shall be considered viable. 

If ChrAVE development proves viable, it may be an 
appropriate platform for further experimentation. Much 
like its cousin the CAVE, the ChrAVE may prove to be a 
valuable platform from which to launch many human-computer 
interface (HCI) experiments. The initial design of the 
ChrAVE was for users performing low-level land navigation. 
As the scope of military helicopter piloting tasks broaden 
for future study, shall the ChrAVE prove durable, if not 
modifiable, for increased demands of such study? 
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Additionally, developing favorably suggests that the 
most beneficial interface has been identified and is 
attainable. To what degree does one need to interact 
physically with the 'near' environment while navigating? 
Grasping and manipulating objects require optics that allow 
proper visual representations of the real world and the 
pilot's hands. Nearly everything in the cockpit 
environment is within arm's reach to the pilot. Should the 
camera optics provide focus and field of view (FOV) that 
properly represent the real world within a meter? In 
Chapter III arguments are presented supporting the 
interface decisions made for the initial implementation of 
the ChrAVE. If these assumptions prove off the mark, can 
they be overcome? 

Lastly, embedding the ChrAVE system into the confines 
of a cockpit may force the development of skills only 
exercised while flying. Cockpit management skills conform 
to the ergonomic demands of the cockpit environment and can 
only be practiced while in such an environment. Might 
being confined by the physical constraints of a generic 
cockpit while practicing navigation aid in the act of true 
real world navigation? Mastering or automating what may 
appear to be a small component skill such as map folding 
and management can prove extremely useful when a pilot is 
applying all his/her mental resources to navigation. Does 
mentally automating the simple aid in resource management 
when tasked with the difficult? Alternatively, will 
attentional demands of component tasks like cockpit 
management decrease the performance of the principle task 
of overall navigation? 
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E. 


ORGANIZATION OF THIS THESIS 


This thesis is organized into the following chapters: 


1. Chapter I: Introduction. This chapter includes 
an introduction to the problem, motivation, and 
outline for the thesis. 

2. Chapter II: Background. This chapter contains 
pertinent background information including a 
summary of the work of Sullivan, McLean, Wright, 
and Vallino, a description of current training 
methods prior to and upon arrival at destine areas 
of flight, and a summary of augmented and virtual 
environments. 

3. Chapter III: Approach. This chapter describes 
the decision process followed to define the goals 
and features of the training apparatus. 

4. Chapter IV: Implementation. This chapter 
describes how the system was implemented. It 
contains descriptions and specifications of the 
hardware components and software employed in the 
implementation. 

5. Chapter V: Methods. This chapter describes 
experimental setup and execution. It provides 
necessary information to recreate the experiment. 

6. Chapter VI: Results. This chapter contains 
results of the experiment. 

7. Chapter VII: Conclusions. This chapter contains 
conclusions reached from the testing process. 

8. Chapter VIII: Future Work. This chapter 
describes the research and implementation ideas 
that the author was unable to perform due to time 
or technology constraints. Additionally, this 
chapter suggests new research questions generated 
by this research. 
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II. BACKGROUND 


A. THE TASK OF HELICOPTER TERRAIN NAVIGATION 

From the early days of flight school each pilot is 
implored to prioritize competing tasks in accordance with 
the 'axiom of Ss', aviate, navigate, and communicate. 
Aviating is the first order of business and requires 
obstacle avoidance and skillful flight control dexterity as 
well as employing proper decision making with regard to 
flight profiles and immediate actions during emergencies. 
Failure to aviate can be fatal. Navigation is attended to 
when aviation is under control. It requires less skillful 
dexterity and is more on the order of planning and 
controlling the course and position of the aircraft. 
Lastly, communication shall be attended to. While simple 
in procedure, communication can rob mental resources from 
aviating and navigating, thereby creating an environment 
that requires correction. Failure to effectively 

communicate is unlikely to be fatal in and of itself, but 
attending to communication while holding aviation and 
navigation at bay can promote their failure. 

Successful helicopter terrain navigation is the 
product of cohesive aircrew teamwork. Each member of the 
aircrew has specific duties and responsibilities. The 
pilot at the controls (PAG) and the pilot not at the 
controls (PNAC) or navigator comprise the main aircrew 
component. Depending on the type of aircraft additional 
aircrew may be aboard and share in the duties of 
navigation; however this thesis will focus exclusively on 
PNAC, the navigator. 
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1. 


Types of Flight Profiles 


This thesis is primarily concerned with flight and 
navigation at or below 200 feet above ground level (AGL). 
The Assault Support Helicopter Tactical Manual (CNO, 1992) 
divides this area of flight into three categories. 

a. Low Level Flight: Flight conducted at a 
selected altitude at which detection and 
observation of the aircraft or of the points 
which, or to which, it is flying are avoided or 
minimized. The flight route is pre-selected, 
generally a straight line and is flown at a 
constant airspeed and indicated altitude. 

b. Contour Flight: Low altitude that conforms 
generally and in proximity to the contours of the 
Earth's surface. It takes advantage of available 
cover and concealment to avoid an enemy's 
observation or detection of the aircraft or its 
departure and landing. It is characterized by 
varying of airspeed and altitude as vegetation 
and obstacles dictate. 

c. Nap of the Earth (NOE) : Flight as close to 
the Earth's surface as vegetation and obstacles 
permit while generally following the contours of 
the Earth's surface. Altitudes and airspeeds are 
selected based on weather, lighting conditions, 
and enemy situation. The pilot preplans a broad 
corridor of operation based on known terrain 
features with a longitudinal axis pointing 
towards his objective, but in flight he uses a 
weaving and devious route within the corridor and 
oriented along the axis to take advantage of the 
cover and concealment afforded by terrain, 
vegetation, and man-made features. 

Depending on many factors, migration from one flight 
profile to another can be instantaneous or even continuous. 
This thesis will primarily deal with low level and contour 
flight; however the task of navigation during the 
experiment will avoid tactical flight and shall embrace 
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navigating along the intended route of flight while hitting 
the intended checkpoints. 

2. Division of Duties 

a. The Pilot at the Controls (PAC) 

The flying pilot is responsible for actual 
manipulation of the controls, avoiding obstacles, and 
reporting key terrain and landmarks to the non-flying pilot 
utilizing standard terrain feature terminology. His focus 
is primarily outside the aircraft. 

b. The Navigator / the Pilot Not at the 
Controls (PNAC) / the Observer 

The pilot not at the controls is referred to by 
different titles amongst the various references; for the 
purposes of this thesis the title navigator will apply to 
the PNAC. The tasks and responsibilities of the PNAC are of 
particular interest to this thesis. The ChrAVE's overall 
system goals are tailored to the needs of the navigator and 
each task has been faithfully emulated for evaluation in 
the experiment phase. The duties and responsibilities of 
the navigator are: 

• Navigating from waypoint to waypoint on the 

intended route. 

• Maintaining orientation / Monitoring location. 

o Associate 3-D terrain with the 2-D map 
representation. 

o Utilize timing as a redundant means of 
monitoring location. 

o Identify/utilizing key terrain features. 
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Checking features. 


■ Channeling Features. 

■ Limiting Features. 

• Provide directional voice commands to the flying 
pilot. 

o Standard directional voice commands, 
o Standard terrain feature terminology. 

• Monitor/manage radios. 

• Monitor cockpit instruments. 

• Manage navigational equipment. 

B. LIMITATIONS OF CURRENT FLIGHT TRAINING METHODS 

Limitations of current training while land based 
methods are mundane except when considering the 1) expense 
of training via actual flight (both monetarily and in 
maintenance man-hours) or 2) access to immersive training 
and rehearsal tools. Standard training techniques employ 
the following: 

• Trainees performing background study of the 
procedure or flight profile 

• Trainees attain rote memorization of the 

procedure 

• Pre-flight brief detailing the procedure or 
flight profile 

• Instructor demonstration of the procedure or 
flight profile in flight 
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• Trainee execution of a procedure or flight 
profile in flight 

• Post-flight brief detailing the trainee's 
performance 

When training is done in this manner via actual flight 
the instructor has competing interests, i.e. maintaining 
situational awareness of all the parameters of the aircraft 
and observing the trainee in total (airmanship and the 
bearing). The instructor can concentrate on observing the 
trainee much more in simulated flight. If the simulation 
is presented to the trainee with high fidelity, the trainee 
will be mentally taxed in a realistic manner and weaknesses 
will surface. This is the case in present full-motion 
instrument flight rules (IFR) simulators; trainees 
monitoring and answering radio calls while monitoring 
instruments, applying flight control corrections, ignoring 
their own proprioceptive system, and maintaining their 
location on an approach plate often demonstrate 
manifestations of being overloaded in a number of ways. 
Breakdowns in airmanship, fixating on instruments, 
unanswered or unacknowledged radio calls, loss of 
situational awareness, faulty cockpit management skills, 
anxiety, agitation, and belligerence are all signs that a 
trainee has encountered a personal training limitation. In 
actual flight, baring airmanship, these manifestations can 
be difficult to observe. The use of helmets and visors can 
compound this difficulty. 

Identifying weaknesses in the trainee's performance is 
only half the battle. Showing the trainee the error of 
their ways and then suggesting alternate techniques that 
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tend to be more successful is the remaining part. It is 
thought that automating some techniques provides more 
mental resources for the remaining competing interests. 

Current training techniques of low-level helicopter 
navigation in simulators do not force the trainee to 
perform the mental calisthenics necessary in actual flight. 
Furthermore, current flight rehearsal systems do not place 
the user in an immersive or compelling environment for 
critical decision-making. 

Lastly, collaborative flight training tools, meaning 
flights of two or more aircrews, simply do not exist 
without actual flight. Traditionally, a land-based 

helicopter squadron only has a single simulation resource 
and if more than one does exists, they are not networked 
for collaborative flight. 

C. LESSONS LEARNED FROM RELATED VIRTUAL ENVIRONMENT 

APPLICATIONS 

It is clear that a deployable trainer must have a 
small footprint. Desktop computer implementations have 
been attempted with varied results. More importantly, 
these implementations have shed light on the value of 
certain features incorporated. It is beneficial to 

leverage these valuable features into any new 
implementation. This statement remains true even if a 
desktop computer implementation is not carried on. 

1. Summary of Sullivan's Research 

Sullivan's work identified helicopter pilots as 
principle subjects in an open terrain navigation 
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experiment. His desktop computer system employed four 19- 
inch monitors, each with a 640 x 480 resolution. The four 
monitors were set up in a semi-circular array configuration 
that provided a wide aggregate 129° field of view (FOV) . 
However, the aggregate FOV was interrupted by monitor 
borders. Center gaze for a user fell upon the center set 
of monitor borders. Sullivan over came this by offsetting 
the viewing frustum 32° to the right. This off-axis 
configuration (Figure 3) allowed for the user's center gaze 
to be coincident with the center of the third monitor. In 
effect, the monitor array mirrors the windscreen array on 
the left side of an actual aircraft since navigation duties 
are normally performed from the left seat in the SH-60F/H. 



Figure 3. From (Sullivan, 1998). Four monitor off-axis 
array. 

Sullivan saw value in employing a large display 
capability. Additionally, Sullivan noted the aggregate 
FOVs of actual aircraft versus motion based simulators, 
TOPSCENE (a single monitor application), and a three- 
monitor configuration of his system (figures 4, 5, & 6) . 
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gray) compared to motion-based trainer (dark 
gray). Adapted from (Sikorsky, 1989). 



Figure 5. Field of view available in SH-60F (light gray) 
compared to TOPSCENE (dark gray). Adapted 
from (Sikorsky, 1989). 

Large display capabilities provide two important 
concepts worth noting. The first is periphery views. When 
there is a large aggregate FOV, the portion of the display 
not being directly looked at moves into the periphery. 
Periphery views not only reflect views available in the 
real world, they also provide significant information to a 
pilot for assessing relative motion of the aircraft. 
Narrow FOV displays allow little, if any, periphery view. 
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Figure 6. Field of view possible using a three-screen 

configuration (dark gray) compared to field of 
view available in SH-60F (light gray). 

Adapted from (Sikorsky, 1989). 

The second concept is that of independent directions 
of gaze and travel. In single monitor implementations, 
such as TOPSCENE, the monitor presents a view married to 
the twelve o'clock position of the aircraft. If the pilot 
wants to view terrain that is on the aircraft's nine 
o'clock, the pilot must maneuver the aircraft so that the 
terrain desired enters the viewing frustum of the 
aircraft's twelve o'clock. However, when the aggregate EOV 
is substantial, the pilot may be able to view terrain in 
the nine o'clock position while maintaining flight in 
another direction. Sullivan noted that larger EOV display 
allows for a longer period of time to view a terrain 
feature without changing course (Eigure 7). 

These two concepts aid in the total immersion 
perceived buy the user. They allow the pilot to interact 
with the VE in a more natural way and they allow the task 
of low-level land navigation to be more realistic. 
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Figure 7. From (Sullivan, 1998). Time available to view 
a terrain feature based on FOV. 

2. Svimmary of McLean's Research 

McLean continued Sullivan's research with emphasis on 
determining if a second-generation system could be used to 
train the tasks of map interpretation and terrain 
association. McLean's implementation employed a three- 
monitor array display. This research yielded proof that 
the system was successful in training the tasks of map 
interpretation and terrain association, however, McLean 
concluded that the system could not be used for testing for 
the tasks of navigation due to lack of any correlation 
between performance using McLean's implementation and 
actual flight. 

Low task realism and system fidelity may have played a 
roll in these outcomes. McLean's implementation was 
effectively a part task trainer for navigation. 
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Specifically, McLean's implementation was strictly 
effective in training map interpretation and terrain 
association without the other additional competing 
interests involved in actual flight. 

Although Sullivan and McLean made no attempt to 
present their systems as cockpit clones, interface 
metaphors do exist. In fact, flight control manipulation 
violates the premise of an aircrew's division of duties for 
the navigating pilot. 

3. Summary of Wright's Research 

Wright recognized that "helicopter missions are never 
defined as '...successful navigation to and return from a 
location.' Navigation, in and of itself, is not the 
mission — it is merely a skill that all helicopter pilots 
are expected to master in order to perform their duties as 
pilots." Additionally, Wright notes that the inherent 
expense and unforgiving nature of helicopter flight, makes 
it a prime candidate for innovative training techniques 
such as virtual three-dimensional (3-D) fly-throughs. His 
research sought ways to assist pilots in planning routes 
for and navigating in an urban environment using 3-D 
graphical displays and virtual fly-through computer models. 
Furthermore, Wright sought to supply the user with the 
graphical fidelity necessary to accomplish the task in 
order to reduce the computational strain on the computer 
system. 

Wright quotes Thorndyke & Golden (1983) on the three 
hierarchical levels of information concerned with 
navigation as it pertains to urban terrain. 
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• Landmark knowledge : information about the visual 

details of specific locations in the 
environment. It is memory for notable 

perceptual features such as uniquely shaped 
buildings. 

• Procedural _ knowledge _ (route _ knowledge) : 

information about the sequence of action 
required to follow a particular route. 
Procedural knowledge is built by connecting 
isolated bits of landmark knowledge into lager, 
more complex structures. 

• Survey knowledge: configurationally or 

topological information. Objects locations and 
inter-object distances are encoded in terms of 
geocentric, fixed, and frame of reference. A 
geocentric frame of reference is a global, map¬ 
like view, while an egocentric frame of 
reference is a first-person, ground-view 
relative to the observer. 

Wright was mostly concerned with providing enough 
visual fidelity for pilots flying into a given urban area 
of flight for the first time. For example, first time 
flight into the Washington D.C. area may include graphical 
depictions of the mall, monuments, and auspicious 
governmental buildings. That may provide enough fidelity 
to determine direction of flight or provide reference 
points for more detailed navigation. 

These levels of navigational information may be 
applied abstractly to navigation or wayfinding other than 
urban flight. Note that while flight in feature deprived 
areas such as at sea, snow-scapes, and desert-scapes can be 
more difficult for determining precise location, general 
location and heading can be relatively simple to determine 
using a minimum or perhaps only two or three reference 
points (key terrain features) . 
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Figure 8. From (Wright, 2000). Aerial Photo vs. Virtual 
Views of Tysons Corner, Virginia. 
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Wright attempts to exhibit notable reference points in 
and urban environment while suppressing the anonymous 
reference points (noise). On a metaphoric level this is 
equivalent to depicting the interesting tree(s) with high 
detail, while suppressing the detail (and polygons) of the 
common trees of the forest. 

D. SUMMARY OF CHROMAKEY TECHNOLOGY 

Chromakey technology has been in existence for a 

number of years in hardware but has been making strides 
more recently in software. This technology identifies a 
key color, often blue or green, that is found in a 
foreground scene and is replaced with the corresponding 
pixel from a background scene. A single composite scene is 
the result. The weather report on the nightly news is a 
prime example of this technology being demonstrated in real 
time. 

Adaptations that are more complicated also exist in 
our daily lives. Figure 8 displays a chromakey technique 
that requires registration so that the placement of the 

virtual or augmented information appears properly aligned 
and fused with the main scene. Note the obscuration effect 

the player has on the virtual yellow line. Also, note that 

there are many shades of green, the key color, on the 
playing surface. A wide key color range is utilized in 
this implementation and it is adjusted continually for 
changes in the videoing environment. However, if the 
player's uniforms were also green, the augmented yellow 
line may obscure a portion of the player. Additionally, the 
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stadium must be mapped out and modeled and each camera's 
placement must be calibrated. 



Figure 9. From (Azuma, 2001). Obscuration and fusion of 
a first down line with moving coverage of the 
play. 


E. AUGMENTED REALITY 

1. Summary of Vallino's Research 


Vallino quotes Aukstakalnis and Blatner 
Virtual Reality (VR) as "a computer generated, 
three-dimensional environment in which a 
immersed." Vallino provides amplifying 

dissecting the term into three parts. 


in defining 
interactive, 
person is 
remarks by 


First, this virtual environment is a computer¬ 
generated three-dimensional scene that requires 
high performance computer graphics to provide an 
adequate level of realism. The second point is 
that the virtual world is interactive. A user 
requires real-time response from the system to 
interact with it in an effective manner. The 
last point is that the user is immersed in this 
virtual environment. 
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Vallino states that "Augmented [RJealty [AR] is the 
merging of synthetic sensory information into a user's 
perception of a real environment." (Vallino, 1998) In 
order to distinguish between VR and AR he refers to 
Milgram's reality-virtuality continuum (figure 8). 


Mixed reality 


Real Augmented Augmented Virtual 

environment reality virtuality environment 

Figure 10. From (Azuma, 2001). Milgram's Reality- 
Virtuality Continuum. 

While arguments as to how an implementation such as 
the ChrAVE should be categorized using this continuum may 
arise, it should be noted that the participation of the 
real world and virtual world varies during execution of the 
many part-tasks performed while navigating. It suffices to 
say that the ChrAVE is a mixed reality system that augments 
a virtual environment (a terrain model) with the real world 
(a cockpit) . 

Vallino creates an overview of the many types of AR 
implementations. His work's main thrust is that the AR 
interface can be made interactive by a form first person 
manipulation. He uses affine representations to define a 
global coordinate system for the induction of virtual 
objects. Manipulation of these fiducial targets directly 
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affects the appearance of the virtual objects in the scene 
with regard to position and orientation (figure 9). 



Figure 11. From (Vallino, 1998). Manipulating Virtual 
Objects. 


Vallino remarks on the necessity of fusion between 
real and virtual worlds for immersion and presence to take 
place in the mind of the user. Fusion requires global 
coordinate system registration so that virtual objects can 
be appropriately rendered in the augmented scene. There 
are a number of ways to provide the computer with this 
information. The two basic ways are 1) for fiducials to be 
optically detected in the real world scene and then 
calculate the position and orientation of the virtual 
object, or 2) for an input device such as a motion detector 
or articulated arm to provide direct position and 
orientation for the virtual. 
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Vallino's implementations augmented the scene with 
virtual objects completely within a single CPU. As a 
result, latency can be a factor in properly rendering the 
scene. In this type of augmentation, the real world is the 
background while the virtual world is the foreground. 
Contrarily, the ChrAVE invokes off-chip chromakey hardware 
that replaces the key color (blue) in the real world scene 
with the augmented scene. This reduces the CPU load in 
rendering the final scene. In chromakey augmentation, the 
real world is the foreground while the virtual world is the 
background. 

2. Summary of Southwest Research Institute's (SwRI) 
Applications 

The Southwest Research Institute (SwRI) has 
implemented a number of trainers demonstrating camera 
tracked chromakey technology. The composite scenes 
presented to the users exhibit scenes where real world 
objects are within a virtual world. The real world is the 
foreground while the virtual world is the background. 

In the infantry implementation (figure 12) the user 
wears a head-mounted display (HMD) that is tracked for 
orientation and position. The HMD is equipped with a 
camera and microphone (figure 11). The user is placed in a 
chromakey blue curtained chamber. The composite image 
presented to the user contains his view of all real world 
objects with a virtual environment invoked as the 
background. In this case the VE is dynamic and contains 
combatants the user must engage. The user's model M-16 is 
also equipped with a tracking device and is fitted with a 
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recoil mechanism. Pointing and shooting the weapon 
registers shots in the VE. Locomotion is provided by a 
pressure plate placed at the users feet. The user wears a 
third motion tracker at the small of the back to provide 
directional orientation to the system. 



Figure 12. SwRI's Camera placement on a modified V8 HMD. 

Since the 'eyes' of the user are provided by a single 
static mounted fixed-focus camera, aiming the weapon using 
the sites in a natural manner is not possible; the weapon 
is best used when fired from the hip. Additionally, 
matching the FOV of the camera and the FOV of the HMD while 
allowing focus of all real world objects prove to be 
difficult; the view of ones hand in the HMD appears larger 
than normal. As a result, hand-eye coordination with real 
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world objects is affected. Performing immediate action for 
a stoppage in the weapon is slow and clumsy when looking at 
the weapon, however, if the user is familiar with the 
procedure it can be performed quickly with the eyes closed. 
To some degree this awkwardness can be reduced with 
extended exposure to the HMD. 



Figure 13. SwRI's Infantry Chromakey Implementation. 

In the High Mobility Multipurpose Wheeled Vehicle 
(HMMWV) implementation two users can use the system. A 
driver position and M-60 machine-gunner position are 
equipped with similar HMD assemblies noted above. The 
driver interacts with the VE by intuitively driving through 
it; the steering wheel, pedals and shifter are input 
devices to the computer system. Meanwhile the machine 
gunner's M-16 is tracked and fitted with a recoil mechanism 
that activates when fired. Once again, the VE contains 
enemy combatants. 
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The desired emergent behavior is that the driver and 
machine-gunner exercise sound crew coordination and 
communication techniques to overcome the enemy. 



Figure 14. SwRI's M-60 mounted HMMWV Chromakey 
implementation. 


While the system does not infringe on the hand-eye 
coordination of the driver, it does impair the gunner in 
some respects. Driving does not require visually 

referencing the steering wheel, pedals, and shifter. This 
system assumes a driver can drive without looking once 
seated; occasional glances are accepted. However, the 
gunner requires a higher degree of hand-eye coordination 
when manipulating and reloading the weapon, which can occur 
often in is such a scenario. As in the infantry 

implementation, sited marksmanship is not possible. But 
with a machinegun single shots are seldom necessary. 
Instead, the HMMWV-mounted weapon is fired and the impacts 
are walked onto the target as in the real world. 

Vallino's implementations augmented the scene with 
virtual objects while relying on CPU power. As a result, 
latency can be a factor in properly rendering the scene. 
In this type of augmentation, the real world is the 
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background while the virtual world is the foreground. 
Contrarily, SwRI's implementations utilize off-chip 
chromakey hardware to create the composite. This reduces 
the CPU load in rendering the final scene, which minimizes 
latency, jitter, and other fusion misgivings. In chromakey 
augmentation, the real world is the foreground while the 
virtual world is the background. Over time Moore's law 
will allow Vallino's implementations to latent-free, 
however, sighting the entire fiducial in a scene 
continually may not be possible do to head movements and 
obscurations. This would result in augmentation 
interruption. Therefore the ChrAVE will utilize an 
implementation similar to SwRI's. 
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Ill. APPROACH 


A. OVERALL SYSTEM GOALS 

The overall goal of the ChrAVE prototype system is 
twofold: 

1) To place the subject in an immersive and familiar 
environment, true in first person fidelity with as few 
physically imposed distractions as possible. 

2) To exercise the task of navigation as faithfully 
and rigorously as the task is in the real world. 

B. DISPLAY DEVICE DECISIONS 

1. Head-Mounted Display (HMD) 

In selecting displays one needs to consider the 
practical and technical limitations of available display 
hardware, as well as the user's requirements and the 
specific tasks in which the display device will be used. 
The trade-off between the display's FOV and spatial 
resolution is a crucial one (Melzer & Moffitt, 1997). 
Sullivan addressed FOV with regard to a single monitor or 
an array of monitors where the direction of gaze is 
dependent a) on the region of view the monitor is intended 
to display and 2) the of direction of flight. All FOVs 
have a coincident point of view or a sweet spot from which 
viewing is optimal for the user. In other words, a monitor 
provides a FOV base off the direction of flight. Pilots 
wanting to inspect specific terrain have to fly in such a 
manner as to place the desired terrain in a region of view. 
While an HMD provides a constant angular FOV, with the use 
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of head-tracking the pilot can dynamically affect the gaze 
independent of the direction of flight. This is regarded 
as a dynamic point of view. Head movements would provide 
all possible views out of the cockpit that the real world 
provides. It would logically be assumed that a head- 
tracked HMD would provide a total FOV equal to that of the 
light gray regions in figures 4, 5, and 6. 

Additionally, HMDs tend to provide a more immersive 
environment for the user. Considering military helicopter 
pilots are accustom to night vision goggles (NVGs) and 
their similarity to HMDs, employing an HMD as the primary 
visual device is the logical choice. 

2. Camera & Lens 

Although binocular vision is used in NVGs, binocular 
vision in the ChrAVE would require duplication of most of 
the signal processing hardware. Although binocular vision 
may prove beneficial in the user's active interaction with 
and manipulation of objects in the real world, such gains 
are expected to be minimal. Since stereopsis is less 
effective beyond 10 meters (Melzer & Moffitt, 1997), 
rendering the VE stereoscopically to the user would also 
provide little to no advantage for VE viewing. Considering 
the early stage of this research and the expected low gains 
in fidelity of a binocular/stereoscopic system, monocular 
vision shall be employed in the ChrAVE. 

The camera will largely comply with two criteria, 
overall system integration and be as high-end as budgetary 
limits will provide. Overall system integration of the 
camera is concerned with 1) lens availability and swap- 
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ability and 2) the video signal requirements that must be 
provided to the remaining components of the system. An 

industry standard mount is most attractive because it 

yields relatively high numbers of potential lenses. 

Lens selection is based off many factors including 
visual requirements such as first-order parameters (focal 
length, FOV, and /-number), performance parameters 

(emphasizing limits of distortion), and other parameters 
(such as size, weight, shape, and zoom). "Commercial 
optics are often useful at the early (budget strapped) 
prototype stage where 'proof-of-principal' demonstration of 
a particular application is important to ensure project 
survival (Fischer, Couture, & McGuigan, 2002) . At this 
early stage of research and considering the research of 

SwRI, it appears that of the shelf optics that optimally 
meet the following parameters shall be considered: 

1) A depth of field within arms reach that renders 
objects in focus and appropriate in perceived size. Depth 
of field depends on three factors, with the size of the 
lens opening, the distance of the objects focused on, and 
the focal length of the lens. 

2) The lens should provide a FOV that matches the FOV 
limits of the HMD. 
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Eye-Lens Displacement (ELD) 


Longitudinal —Z 
Vertical y 
Pitch Angle -a 


Figure 15. Eye-Lens Displacement. 

3) Size, weight, and shape are considered because the 
camera and lens will be mounted onto the HMD. Size and 
shape will dictate possible locations to mount the camera 
and lens onto the HMD. It is desire for the user's eyes 
and the camera to share the same optical path. Since that 
is rather complicated for this early research, minimizing 
the collective separation of these paths shall be 
emphasized. The camera and lens must be mounted in such a 
manner as to minimize the displacement of the camera to the 
user's eyes yet still converge on an acceptable place in 
front of the face so the user can easily refer to their 
hands. Parallel optical paths would have the user reaching 
too high or low or initiating excessive head movements to 
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see their hands. This eye to lens displacement (ELD) 
represents both a rotation and translation between the 
user's and camera's optical path origin (figure 14) . ELD 
is expected to affect the user's active interaction with 
and manipulation of objects in the real world environment. 
The weight and balance of the HMD upon the user's head 
effects both rate of fatigue and the user's interactive 
information gathering. The entire head assembly will mimic 
the physical demands already placed on helicopter pilots 
employing NVGs. 

Although methods exist that can minimize ELD to near 
zero by employing cameras and mirrors assemblies, such 
methods have not be explored due to their expected 
sophistication, fragility, expense, and the aforementioned 
size and weight issues. The advantages that such methods 
may have would be rather interesting to study in follow-on 
work. That said, Wurpts notes that Biocca and Rolland 
"found that the displacement of the camera position from 
the eyepoint can cause an inter-sensory conflict between 
the human visual and kinetic senses.' This conflict 
increases the difficulty that users experience when 
interacting with real objects while immersed in the virtual 
environment." 

C. MOTION TRACKER DECISIONS 

A motion tracker that provides 6-DOE, position, and 
orientation is determined to be required, although one 
could argue that only orientation is essential since head 
movements in the near field would be notice via the 
camera's view of the real world, and that the far field 
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would simply not be noticed because the virtual terrain 
viewed is too far away to pick up any discernable angular 
difference. As needed, location tracking can be turned off 
only allowing for orientation. A line of site motion 
tracking system may be vulnerable to the movements of the 
user, therefore, position detection with inertial 
redundancy is desired. Also attractive is a tracking 
device that imposes minimal movement constrictions on the 
user. 



Figure 16. Modified from (InterSense, 1999) IS-600 Mark 2 
block diagram. 
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Figure 17. From (InterSense, 2001) IS-600 Mark 2 

D. LOCOMOTION DEVICE DECISIONS 

During flight, the navigating pilot will direct the 
flight of the aircraft by giving appropriate voice commands 
to the pilot at the controls (the proctor) . See Appendix 
B, page 5 of the ChrAVE Experiment Questionnaire for a list 
of navigational commands available to the navigator. This 
behavior is identical to that utilized in actual flight, 
therefore, no learning curve exists for the subject. 

Such navigational behavior can be characterized as 
active in terms of mental activity required to effectively 
determine position, course, and the next verbal command 
while navigating, but passive in terms of actually 
manipulating the flight controls. Ironically, the 
navigator is generally mentally more 'active' than the 
pilot at the controls. While the pilot at the controls has 
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a view of the outside world, he may not know where he is in 
that world. Contrarily, the navigator actively compares 
the map to the real world. Since the navigator has the map 
as a reference to all the terrain in the area of flight it 
follows that he is more active in the investigation of that 
terrain. In fact, while it is a crew coordination task to 
maintain situational awareness and knowledge of the 
aircraft's whereabouts at all times, it is the navigator, 
who through use of the map and the outside world challenges 
the certainty of the aircraft's place in space. Successful 
navigation requires vigilant uncertainty management, the 
degree to which uncertainty is minimized and considered 
acceptable. 

Figure 18 suggests that being both the PAG and the 
navigator would more heavily task a single pilot than 
employing a division of duties amongst the two pilots. 

Because conversational, real world voice commands are 
to be used, computer recognized voice commands were 
abandoned; there are simply too many verbal ways to direct 
the same activity. Voice commands recognized by computer 
require that they be precisely structured and 
miscommunication (rate of error) would be far too high to 
be reliable with the complexity of phrases and utterances 
made by the navigating pilot. 
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Figure 18. Passive Active Matrix. 

Actual manipulation of the ChrAVE's flight model will 
be done by the proctor via keyboard. Each basic command 
will be accomplished by providing the appropriate 
corresponding keystroke. Utilizing canned keystrokes 

instead of a joystick type of input device ensures that 
each subject participating in the experiment receives 
identical response to their commands. Eor example if a 
subject calls for a right turn to 9 o'clock, a keystroke 
initiates that turn. Each turn is identical in terms of 
roll in/out acceleration and accuracy. 
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E. MOCK COCKPIT DECISIONS 

The ChrAVE primitively mocks the right half of a side- 
by-side dual piloted helicopter. It is meant to be rather 
generic to all helicopter communities. It employs a semi¬ 
surrounding wall, roof and windscreen panes to provide 
immersive visual aesthetics. They are designed to be 
realistic obstacles that cause real world obscurations and 
can be referenced to determine rate of movement and 
attitude with regard to the viewable virtual world. These 
structures impede line of sight and force head movement 
during the task of navigation. 

An instrument panel includes an airspeed indicator, an 
attitude indicator, an altimeter indicating height above 
mean sea level (MSL), a turn rate indicator, a compass, and 
a vertical speed indicator (VST). 

The flight controls, a cyclic, a collective and rudder 
pedals are also employed to be normal helicopter obstacles 
for the navigating pilot since a navigating pilot does not 
use these input devices while navigating. 

F. FEEDBACK 

The navigator, through use of the HMD, will have a 
merged view of the real world and the virtual world. The 
real world shall consist of the mock cockpit, objects 
within that cockpit, and the navigators view of himself. 
The virtual world shall consist of a computer-generated 
world complete with images of terrain, buildings, aircraft, 
ground vehicles and atmospheric effects for the specified 
area of flight. Head movements will affect viewpoint 
changes in both the real and virtual worlds. Head 
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movements will allow natural interaction/investigation with 
both worlds . 

The instrument panel also provides vital feedback for 
the navigation of the aircraft. The compass is most 
important in determining heading, while the attitude 
indicator assists in determining the aircraft's orientation 
relative to a virtual world axes. The turn rate indicator 
provides information about the present flight profile. The 
VST determines whether the aircraft is in a climb or 
descent. These instruments, cross-referenced with an 
inside-out view of the virtual world, provide necessary 
feedback that mimics the feedback of navigation during 
actual flight. However, since the ChrAVE system is a 
motionless platform there is a mismatch between visual 
perception and both vestibular and proprioceptive percepts. 

This type of conflicting information is not new to the 
subject pool. Quite often throughout ones aviation career 
a pilot is placed in a position where their visual 
perceptions conflict with their vestibular and 
proprioceptive percepts. This can occur when flying either 
full-motion or motionless simulators, or during actual 
instrument flight rule (IFR) flight. In the simulator 
examples there is an inherent mismatch between the motion 
one expects, feels, and sees both in their view of the 
world and in their instruments. This occurs more during 
simulated day visual flight rule (VFR) conditions because 
visible reference points underscore the mismatch between 
views of the world and the motion administered to the user. 

Conversely, when simulating day or night IFR 
condition, the outside world is featureless; the aircraft 
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pitches and rolls without noticeable change to a cloud or 
dark obscured view of the world. In fact, focusing on 
overcoming the effects of such conflicting information 

makes one trust in and rely on their instruments more 
effectively; seasoned pilots have been trained to trust 
their instruments over their seat of the pants feeling. 

Fidelity in this research is emphasized in some areas 
while all but ignored others. The mock cockpit, for 

instance, provides a metaphor for normal cockpit 
interaction, but has little similarly to any actual 
aircraft. Yet in all circumstances fidelity adheres to 

this definition: the degree to which a system accurately 
reproduces the sensory experience of its real world 
counterpart, often with minimal intended distortion. This 
includes fidelity with regard to visual, auditory, 
olfactory, gustatory, tactile, vestibular, haptic, 
proprioceptive, etc. percepts. 

It may be said that a system's collective fidelity ^ 
the system's interface. The interface and the feedback 
provided produce the system's collective sensory perception 
in the user's mind. Large sensory experience distortions 
reduce immersion, presence and may infringe on the 
intuitive nature of the system's input device. For 
example, flying with a mouse vice collective and cyclic 

places an unnatural burden on the user to 'learn' the 
system. In some instances, this can be considered negative 
training. A system that provides a sound sensory 

perception without noted distortions is assumed to place 
the user's mind in a realistic setting, replete with 
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stress, competing interests, and myriad subtasks often lost 
in lower fidelity systems. 

G. POTENTIAL OPERATING MODES 

There are numerous modes of operation the ChrAVE 
system can be used for, each of which may spawn its own 
research questions and techniques for understanding 
psychology and the potential of training via an embedded 
trainer. 

In an instructor-student mode, navigational and 
cockpit management techniques can be monitored and coached 
with the undivided attention of the instructor. 

As a route rehearsal tool, a route of flight can be 
planed and practiced, thereby providing the navigator with 
an acquired spatial knowledge of that area of flight 
without ever having actually flown there. 

In a limited air-reconnaissance role the user, 
possibly ground combat personnel, can use it to investigate 
the lay of the land, key points of terrain, possible 
avenues of approach or departure, and lines of sight or 
obscuration levied by the terrain. 
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IV. IMPLEMENTATION 


A. HARDWARE AND PHYSICAL SETUP 

The current ChrAVE system was developed as a practical 
intermediary step in establishing the viability and 
usability of embedded trainers. The ChrAVE acts as a 
laboratory from which to launch research into the 
psychology and the potential of training certain tasks via 
trainers/simulators. The ChrAVE primitively mocks the 
right half of a side-by-side dual piloted helicopter. It 
is meant to be rather generic to all helicopter 
communities. 

1. Platform 

The platform is comprised of a deck, pilot seat, 
flight controls, and some surrounding structures that 
emulate the walls, windscreens, and overhead of the cockpit 
(figure 19). 

a. Seat & Flight Controls alignment 

The current implementation used a Elight Link 
Inc. seat and basic helicopter flight controls. These 
controls mimic standard multi-axes game port input devices 
to PCs. Two axes (pitch & roll) are dedicated to the 
cyclic, one (thrust) to the collective, and one (yaw) to 
the rudder pedals. Additionally, there is a button on the 
collective that can be given specific assignments. The 
flight controls were not used by the navigating pilot 
during the experiment. They only provided aesthetically 
realistic obstacles to the task of cockpit management for 
the navigating pilot. 
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Figure 19. The ChrAVE Platform. 



Figure 20. From (Flight Link Inc., 2001) Rotary Wing 
Hardware. 
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b. Cockpit Wall, Roof, and Windshield Panes 


The semi-surrounding walls, roof and windscreen 
panes used in this implementation provided immersive visual 
aesthetics. These too, were designed to be realistic 
obstacles. These structures impede line of sight and force 
head movement during the task of navigation. Additionally, 
the deck was specifically designed and built to allow sight 
through a mock chin bubble. A chin bubble is a windscreen 
that provides sight down from the aircraft. 

c. Instrument Panel 

The instrument panel presentation is run on an 
SGI LCD monitor. Packets are sent to the instrument panel 
computer from the VE computer. Each packet includes the 
necessary information to drive each of the instruments. 
OpenGL and Visual Studio 6.0 are used to create network 
connectivity and run the graphics engine. The LCD screen 
provides for flicker-free viewing with the camera. A 
conventional cathode ray tube (CRT) monitor would have to 
either be equipped with a sync gen or boost the refresh 
rate 
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Figure 21. ChrAVE Instrument Panel (SGL LCD monitor). 


2. Headgear 

a. Head Mounted Display 

The Head Mounted Display (HMD) selected maintains 
a high standard in performance among professional HMDs, 
even though its active matrix Liquid Crystal Displays (LCD) 
have a Video Graphics Array (VGA) pixel resolution of 
( (640x3)x480) . Considering cost versus performance, HMDs 
of higher resolution were far too costly for this research. 
The V8 provides a CRT quality image. The V8 allows for 
interpupillary adjustments as well as eye relief 
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adjustments. The V8's earphones were not used during this 
research therefore they were rotated away from the ears 
above the headband. Audio was provided by a surround sound 
speaker system detailed later. See specification sheet A in 
Appendix A. 



Figure 22. From (Virtual Research, 2000) V8 HMD. 

Inputs and outputs for audio, video, and power 
are handled through an external control box. Red Light 
Emitting Diodes (LED) indicate 'Power On' and 'Stereo' 
modes. Standard 15 pin VGA type connectors accept VGA (640 
X 480 60Hz) inputs, readily available on today's graphics 

engines and workstations. 

ib. Camera 

The camera used in this implementation was an 
Auto Gain Control (AGC) and Electronic Light Control (ELC) 
Panasonic with three Charged Couple Devices (CCD), one each 
for red, green, and blue. See specification sheet B in 
Appendix A. 
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Figure 23. From (Matsushita Electric Corporation of 

America, 2002) Panasonic 3CCD Color Camera 
Head (GP-US532H) and CCU (GP-US522CU). 

c. Lens 

The camera lens used was a fixed focal length 
(4mm) lens. It has two adjustment rings. One is for focus 
and the other is for aperture f/stop settings. Changing 
the aperture to a lower f/stop # allows more light to reach 
the camera sensors but it reduces the depth of field. See 
specification sheet C in Appendix A. 
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Figure 24. Photo of ChrAVE headgear. 

d. Motion Tracker 

The IS-600 Mark 2 was used in this 
implementation. It is a hybrid motion tracker that 

utilizes inertial and ultrasonic sensing technologies to 
provide 6-DOF. The Mark 2 provides multimode communication 
redundancy for the inertial and ultrasonic hybrid 
components. The inertial system is comprised of an 
InertiaCube™ that is strapped to the user's headgear and 
tethered by wire from to the control unit. It is nearly 
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immune from environmental interference. The ultrasonic 
system is comprised of SoniDiscs™ place adjacent to the 
InertiaCube™ on the user's headgear and an X-bar installed 
overhead. The SoniDiscs™ chip an ultrasonic burst when 
they sense an infrared flash from the X-bar. The X-bar is 
equipped with microphones on each of the four pods. When 
the X-bar hears the ultrasonic chirp on the four pods the 
location of the SoniDisc™ is calculated by the control 
unit. The SoniDiscs™ are more susceptible to interference. 
They require line of sight communication and normal indoor 
environmental light intensities do to the infrared portion 
of the system. See specification sheet D in Appendix A. 



Figure 25. IS-600 Mark 2 X-bar suspended from ceiling. 
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3. 


Chromakey Bluescreen Matting 


A backdrop made of standard entertainment industry 
chromakey blue cloth panels was constructed in such a 
fashion so as to surround the mock cockpit from the eleven 
o'clock to the four o'clock. Where necessary, chromakey 
blue tape was used to hide seems. 



Figure 26. From (Mole-Richardson Co. Inc., 2001) 2-inch 
wide chromakey blue tape. 

4. Lighting 

Lighting is by far the most temperamental component to 
implementing chromakey technology. The chromakey mixer 
must perceive the chromakey blue backdrop (called the 
matting) without noise such as being unevenly lit and 
having shadows. A number of fluorescent lamps were placed 
about the mock cockpit in such a manner so as to light the 
matting while not impeding the navigator's view of the 
matting. An additional hurdle was ensuring that the lamps 
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did not directly shine into the camera lens or the sonic 


disks. Although the sonic disks are alerted to infra 
light, the intensity of the fluorescent lamps can create 
sufficient noise to disrupt proper motion tracking. 

This implementation employed four fixtures that were 
four feet in length and four fixtures that were two feet in 
length. 



Figure 27. From (Flo Co, Inc., 2001) Solo Fluorescent™ 
Lamp. 


Each fixture had high output flicker-free ballast that 
operated on 120 VAC/60Hz. Each fixture also included a 
specular reflector, and two lamp barn doors. 

5. Signal Converters and Mixer 

A number of signal converters were used in the system. 
The system demands that signal quality and integrity be 
maintained throughout the video pipeline. The Ultimatte™ 
400 Deluxe chromakey mixer was the cornerstone used in this 
implementation and required a CCIR-601 signal as input. 
Therefore, both the foreground (EG) signal (an RGB signal 
from the camera) and the background (BG) signal (a VGA 
signal from the CPU) had to be converted. furthermore, 
once the EG and BG signals were mixed, the CCIR-610 output 
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signal had to be converted back to a VGA (640X480) signal 
for the HMD. 


Chromakeyed Augmented Virtual 
Environment Hardware Schematic 
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Figure 28. Schematic of the ChrAVE System. 


a. Virtual Environment VGA to Digital 601 
Signal Converter 

In order to accommodate the Ultimatte™ 400 Deluxe 
chromakey mixer's input signal demands, an Extron™ 
Electronics VSC 200D video scan converter was used to 
convert the virtual environment CPU's 15-pin VGA video 
signal into a digital CCIR-601 signal. See specification 
sheet E in Appendix A. 
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b. Camera RGB to Digital 601 Signal Converter 

The Panasonic™ GP-US542 3-CCD High Performance Color 
Camera produces an RGB signal that must be converted to a 
digital CCIR-601 signal for input to the Ultimatte™ 400 
Deluxe chromakey mixer. The Leitch™ ADC-6801 signal 
converter serves this purpose. See specification sheet F in 
Appendix A. 

c. Chromakey Mixer 

The Ultimatte™ 400 Deluxe chromakey mixer takes 
two digital CCIR-601 signals (a camera feed and a CPU VE 
feed), merges them into a single video image using 
chromakey technology, and outputs the resulting digital 
CCIR-601 signal. See specification sheet G in Appendix A. 

d. Digital 601 to VGA Signal Converter 

A Leitch™ SDC-100 serial digital to VGA converter 
was employed to return the digital CCIR-601 signal from the 
Ultimatte chromakey mixer to a useable signal for the V-8 
HMD (VGA 640X480). See specification sheet F in Appendix 

A. 
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Figure 29. Photo of ChrAVE cart. 
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B. SOFTWARE 

The software selected and used on the VE CPU in this 
implementation cornerstoned MultiGen-Paradigm's Vega 
virtual environment software. Vega features a fairly 
intuitive API application called Lynx that allows 
connectivity between objects (observers, models, terrain, 
effects, etc.) . The VE CPU broadcast packets to both the 
instrument panel CPU and the top-down view CPU. All three 
computers employed OpenGL. Microsoft Visual C++® 6.0 was 
also installed on all three CPUs. It served as the 
platform upon which OpenGL and Vega ran. 

Models and terrain were created using MultlGen- 
Paradigm's Creator software. This software allows for 
importation or creation of geometric models as well as 
textures for mapping onto the models. 
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V. 


METHODS 


A. EXPERIMENT SETUP 

1. Subject Pool 

This experiment used fifteen designated military 
helicopter pilots. All subjects were male students of the 
Naval Postgraduate School (in a non-flying status) and were 
either U.S. Marine Corps or U.S. Navy pilots. Since all 
subjects were designated pilots they meet the expert 
criteria with regard to the knowledge about, and skills 
involved in, the activities of a multitasked cockpit 
environment. Dynamic prioritization of the tasks at hand 
is a critical quality required for all helicopter flight 
regimes. 

2. Treatment 

Each subject's participation involved an entrance 
questionnaire, followed by map preparation for the route of 
flight, a battery of physiology tests prior to flight, the 
low-level navigation flight, a battery of physiology tests 
following the flight, and an exit questionnaire. Lastly, 
each subject was asked to evaluate the performances of 
their peers. 

a. Entrance Questionnaire 

The questionnaire, which can be found in Appendix 

B, briefly asks about medical history, flight experience, 
and parameters for conducting acceptable low-level 
helicopter navigation. A series of slides were shown to 
each subject depicting an intended route of flight with 
checkpoints and a fictitiously flown flight path. 
Additionally, fictitious estimations of where checkpoints 
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were located were also depicted. Subjects were asked to 
evaluate each slide as either acceptable (pass) or not 
acceptable (fail). This provided a baseline estimation of 
acceptable performances across all the helicopter 
communities represented. The slides each subject was to 
evaluate can be found in Appendix B. They were randomly 
lettered and presented to the subject in no specific order. 

b . Tasks 

The tasks each subject was to perform can be 
found in Appendix B (questionnaire pages 4-5) . Each task 
was included in order to provide the subject with a 
realistic navigational workload. 



Figure 30. Preparing a map for an intended route of 
flight. 
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Since we cannot compare ChrAVE results to results 
of actual flight due to the unavailability of real aviation 
resources, the opinions of actual pilots tasked with 
realistic workloads appears beneficial in partially 
determining whether a system such as the ChrAVE is viable 
as a laboratory for continued training in virtual 
environment experiments. 

The subjects were provided necessary resources 
(Appendix B, (questionnaire page 4)) including scissors and 
tape to completely prepare their map for the intended route 
of flight (figure 30). The subjects were additionally, 
provided flight parameters (Appendix B, (questionnaire page 
4) ) in order to correctly prepare their maps with regard to 
time checks and to establish a mindset for the tempo of the 
flight. There was no time limit in map preparation or map 
study. Map preparation and map reconnaissance are initial 
steps use with 2D familiarity of an area. 

c. Physiology Tests 

A battery of physiology tests were administered 
four times throughout the experiment. 

1) The unhooded baseline battery was 
conducted just prior to the subject donning the ChrAVE 
headgear. It provided a baseline of the subjects at rest 
from which to measure any future degradation. 

2) The initial HMD exposure battery was 
conducted immediately following the donning of the 
headgear. 

3) The extended HMD exposure battery was 
conducted following the flight portion of the experiment. 
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4) The unhooded recovery battery was 
conducted immediately following the removal of the 
headgear. 

In each battery there were four tests: 

1) Visual acuity test was intended to show 
any apparent degradation in the ability to read writing on 
the map while hooded. A simplistic eye chart (figure 31) 
created in Microsoft Word consisting of lines of random 
letters. The lines of letters had point sizes ranging from 
50 points to 8 points. All letters were in the courier 
font. Subjects were handed the chart and asked to read the 
smallest line possible. Subjects were allowed to present 
the chart as close to their eyes or camera as needed. 

2) Color identification test was intended 
to show any apparent degradation in the ability to 
correctly perceive colors on the map while hooded. Once 
again, the eye chart in figure 31 was utilized. Six lines 
(blue, red, green, orange, purple, and black) with a width 
of three points were depicted. Subjects were handed the 
chart and asked to state the perceived color of each line. 
Subjects were allowed to present the chart as close to 
their eyes or camera as needed. 
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Figure 31. Visual Acuity and Color Identification Eye 
Chart (Actual size not depicted.) 



Blue 

Red 

Green 

Orange 

Purple 

Black 

R 

0 

255 

0 

255 

128 

0 

G 

0 

0 

255 

102 

0 

0 

B 

255 

0 

0 

0 

128 

0 


Table 1. R-G-B values use to define the colors in the 

color identification test 
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3) Dvorine psudo-isochromat ic plates were 
intended to determine the extent to which the ChrAVE 
introduces color blindness to subjects wearing the HMD. 
Subjects were handed the chart and asked to state the 
perceived number on each plate. Subjects were allowed to 
present the plates as close to their eyes or camera as 
needed, however the plates had to remain at a right angle 
while being viewed. Additionally, subjects were not 
allowed to trace the number on the plate. All other 
established administration procedures were enforced. 
Delays of more than five seconds resulted in an 
identification failure for that plate. 

The Dvorine Color Vision Test consists of a bound set 
of color plates. These plates feature a number or design 
made up of colored dots against a background of contrasting 
dots. The figures are easily identified by persons with 
normal vision, but not by those with color blindness. 

4) Hand-eye coordination test was intended 
to show any apparent degradation in the ability to interact 
physically with the real world. The test entailed the 
subject sitting in a chair three feet from the proctor who 
was also sitting in a chair. The proctor would toss a ball 
to the subject ten times. The subject was supposed to 
catch the ball using one or two hands. A legal toss 
consisted of an apex never higher than the subjects head 
and never lower than the subjects shoulders. Each toss was 
graded as a catch (no discernable fumbling), a fumble 
(fumbling, but not dropped), and a drop. 
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Figure 32. Hand-eye coordination catch test, both 

unhooded and hooded with the ChrAVE head 
assembly. 


d. Virtual Navigation 

A navigational warm-up lasting approximately 
three minutes was conducted. During this period, subjects 
were exposed to turns to the left and right using both 
standard and half standard rates. The proctor verbally 
made note of the length of time it took to roll into and 
out of these turns. Additionally subjects were instructed 
to provide the necessary verbal commands to fly 
perpendicular to a road and then turn left so as to align 
the aircrafts flight along the road. This drill displayed 
two things to the subjects, 1) that the turning radius of 
the ChrAVE was rather wide and 2) that turns to the left 
limit the navigator's view. 

Following the warm-up, the subject was suspended 
in space at the course entry point and allowed to establish 
their orientation using the compass, map, and the available 
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views. When the subject was ready, the sound file was cued 
and the aircraft began to fly. 


Subjects had to listen for radio calls; calls 

correctly identified and responded to were noted by the 
proctor. Periodically demands for the subjects to plot 
their position and orientation were issued; these calls 
came about every two minutes. Additionally, subjects had 
to provide navigational instructions to the proctor. 

This flight lasted 30 minutes. In that time, the 
subject was supposed to negotiate as much of the course as 
possible. The proctor only provided guidance by form of 
cardinal heading, if the subject was hopelessly lost or 

about to fly out of view the top-down viewing monitor or 
off their paper map. 

e. Exit Questionnaire 

The exit portion of the questionnaire (Appendix 

B, (questionnaire pages 9-11)) was presented to the 

subjects following the unhooded recovery battery of 
physiology tests. 

f. Debrief 

After completion of the questionnaire, the 
subjects were invited to view their performance on the top- 
down viewing monitor. The proctor would point out 

observations and key points during the flight. The flight 
path and position plots of each subject's performance is 
depicted in Appendix C. 

g. Subject Peer Evaluations 

Following the subject pools navigational efforts, 
each member of the subject pool was presented with the top- 

70 



down vies of the flight path and position plots of the 

entire subject pool. They were asked to evaluate each 
ChrAVE performance on the following three criteria: 

• The subject's ability to maintain a flight path in 
acceptable proximity to the intended path. 

o Considering the Fort Irwin terrain, evaluate the 
subject's ability to fly the intended route and hit 
the checkpoints. The intended route and checkpoints 
are green while the subject's flight path is red or 
yellow. 

o Rate the performance using a 1 to 7 scale, 

indicating highly acceptable while '7' indicates not 
acceptable. 

o This criterion is independent of the following 
criteria, meaning the proximity to the intended 

flight path is to be evaluated independently of 

whether or not they knew where they were. 

• The subject's ability to correctly estimate their 
location. 

o Considering the Fort Irwin terrain, evaluate the 
subject's ability to accurately locate and plot his 
position (including heading) on demand. Aircraft 
icons of matching color help to pair a subject's 
estimated and actual locations. Note the icon 
outline color. Where necessary, white lines help 
group the pairs. 

o Rate the performance using a 1 to 7 scale, 

indicating highly acceptable while '7' indicates not 
acceptable. 

o This criterion is independent of the preceding 
criteria, meaning the accuracy of the position 
estimation is to be evaluated independently of 
whether or not they were on the intended route. 

• The overall performance. 

o Rate the overall performance as acceptable or not 
acceptable ('A' or 'N' ) 
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The results of these peer evaluations are listed in 
Appendix E. 

B. KNOWN ARTIFICIALITIES 

The ChrAVE system incorporated many known 
artificialities; some were desired, some were not. 

1. Physical Perceptions. 

a. Optical Artificialities 

Real world color is presented to the subject with 
minor deviations. These color deviations occur along the 
foreground or real world pipeline. This pipeline consists 
of the camera, RGB to digital 601 signal converter, the 
chromakey mixer, the digital 601 to VGA signal converter, 
and the HMD. Most likely the HMD has the greatest effect 
on perceived color deviations to the user. The mixed 
signal going in to the HMD is looped through to a monitor 
for the proctor's use. This monitor displays less color 
deviation than the HMD proving that the pipeline prior to 
the HMD has less of a contribution to color deviation. 

The camera is equipped with an automatic gain 
which adjusts the brightness level of the camera's signal. 
The HMD is most effected by the automatic gain adjustments. 
The user can perceive alternating periods of real world 
brightness and darkness with rapid head movements that go 
from repeated head down to head up. 

The lenses of the HMD introduce astigmatism to 
the user. This is most noticeable with large head 
movements while viewing an un-augmented real world view; 
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the astigmatism is almost undetectable while viewing mixed 
real world and VE signals. 


The camera lens had a fixed focus range that 
allowed objects within arms reach to be in focus while more 
distant objects were out of focus. For the most part all 
items that had to be in camera focus were within arm's 
reach. A blurred blue screen actually aided in smoothing 
any shadows or wrinkles captured the keyed matt. 

The static FOV of the HMD was 60° on the diagonal. 
Such a small FOV did not allow for a periphery view being 
presented to the user. Users commonly compensated for this 
narrow FOV with extra head movements. 

The aggregate VE FOV was limited to the coverage 
of the blue screen. Its coverage was approximated the 
user's eleven o'clock to the four o'clock. The user is 
more impaired while looking across the cockpit than in the 
real world. 


b. Auditory Artificialities 

During actual flight a pilot will hear varying 
sounds as the rotors beat the air during climbs, descents, 
and turns. Additionally, a pilot would hear an electrical 
culmination of engines, radio static, and other electrical 
equipment fed to him over the internal communications 
system (ICS). These also vary during different flight 
maneuvers and equipment usages. 

During the virtual flight phase of the experiment 
subjects were exposed to a 30-minute helicopter recording 
that a pilot might hear. Rotor sounds did not vary with 
climbs or descents, nor did the electrical ICS sounds. 
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Random radio calls were included. Additionally, radio 
calls to the subject's aircraft and about the subject's 
aircraft were included. Calls to the aircraft were 
expected to be acknowledged. Lastly, approximately every 
two minutes the subject would here a beep followed by the 
words "plot your position now". 

(1) Radios and Volume control 

The recording was comprised of engine whine. 


rotor thumping. 

and radio calls. It 

was 

not 

possible 

for 

the subjects to 

increase the 

volume 

of 

the 

radios, 

like 

they might in 

actual flight. 

without 

increase all 

the 


helicopter sounds as well. 

Radio calls may have been scratchier than in actual 
flight. 

(2) Helicopter Noise (Bass) 

Headphones were not use. The recording was 
presented to the subject via five speakers, two in the 
front, two in the rear, and a sub-woofer placed behind the 
seat of the subject. Bass of the recording was presented 
to the subject in a louder then normal representation to 
provide the feel of aircraft vibration. 


c. Vestibular Artificialities 

The ChrAVE does not incorporate a motion 
platform. Therefore, there will be obvious intersensory 
conflict between physically sensed motion and visually 
perceived motion. However, since the ChrAVE is intended to 
be a deployable training system, it is important to 
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recognize vestibular differences between the ChrAVE and 


actual flight with both a land based and sea based ChrAVE. 

(1) Non-motion (land/docked) 

The intersensory conflict of a land based ChrAVE 
resides between visually perceived motion and the absence 
of any physical motion. 

(2) Unsynchronized motion (at sea) 

The intersensory conflict of a seabourne ChrAVE 
resides between visually perceived motion and the sensed 
physical motion dictated by the current sea state. 
Although a seabourne ChrAVE was not used in this experiment 
this statement is made with relative certainty. 

2. Ergonomic Artificialities 

a. Generic Airframe 

The generic cockpit was metaphoric in nature to 
any actual helicopter, and did not reproduce any ergonomic 
features of any aircraft with high fidelity. 

ib. Instrument Panel 

A generic navigational instrument panel was 
established which incorporated necessary instruments of 
low-level navigation (altimeter, magnetic compass, attitude 
indicator, turn indicator, vertical speed indicator, 
airspeed indicator) . 

c. Size of Instruments 

The instruments were rendered larger than normal 
to overcome viewing difficulties inherent with the selected 
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camera lens. Also the blue sky of the attitude indicator 
was replaced with orange to avoid chromakey blue matching. 

d. NVG / HMD Difference 

The HMD was reported by many subjects to be 
similar to NVGs. However the HMD has a baffle that does 
not allow site outside of the HMD, whereas NVG allow below, 
left and right of the device. Peripheral site is not 
available in the ChrAVE with the HMD's current 
configuration. 

e. Helmet / HMD, Tracker, Camera 

The weight of the camera, HMD, and worn tracker 
components was comparable to that of actual flight however 
the balance was not. The ChrAVE's head gear weight resided 
almost entirely in the front, thereby applying bothersome 
pressure to the bridge of the nose and fatigue to the back 
of the neck.. 

f. Clock and its Familiarity 

The clock provided to the subjects was not a 
typical timing device purchased by aviators. Nor was the 
clock similar to the standard seven-day clock found in 
naval aircraft. The clock counted down to zero as opposed 
to counting up to a specific time of a given leg. This 
complicated quickly ascertaining the aircraft's progress of 
a given leg. Seeing say, 0:45 on a countdown of a 60 
second leg is different from seeing the normal display of 
0:15 seconds of elapsed time the subjects were use to. 
This was not intended; it was an oversight. 
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3. 


Flight Profile Artificialities 


The subject pool did not have a single aircraft in 
common. The ChrAVE attempted to accommodate all helicopter 
pilots with a single generic set of flight characteristics. 

a. Airspeed 

Airspeed was pegged at 90 knots and did not vary 
in any flight profile. 

b. Windless Day 

All flight was conducted in a windless 
environment. Therefore airspeed matched groundspeed in all 
directions. This simplified and isolated performance with 
regard to timing distances while navigating. 

c. Constant Zero Pitch 

The ChrAVE maintained it's nose on the horizon in 
all flight profiles. This minimized the possibility of 
disorientation due to varying pitch. 

d. Rotor Head Chops Off Special Effects 

An anti-aliasing shortcoming that could not be 
overcome resulted in the virtual spinning rotor head, which 
was visible to the users, chopping off any environmental 
special effects. Eor instance, a plume of smoke that rose 
up from the ground should have been seen as it extended 
beyond the spinning rotor blades. It did not. 

e. Rate of Turn 

Again, in order to avoid the possibility of 
disorientation due to a tight rate of turn the ChrAVE made 
turns that were much wider that the subjects were use to. 
In fact, rate of turn may have presented the greatest 
adjustment to normal flight for the subjects. 
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f. 


Limited to 1/2 SRT or Full SRT 


The ChrAVE either flew wings level, in a half 
standard rate turn (3° of heading change per second with 22° 
of angle of bank, or a full standard rate turn (6° of 
heading change per second with 45° of angle of bank. 
Transition between these profiles was automated with an 
algorithm that mimicked normal acceleration into and out of 
all turns in the roll axis. Doing so provided a smooth 
realistic platform for navigation while minimizing the 
user's disorientation and disruption of immersion. 

4. Tasks Artificialities 
a. Map Prep 

A few map preparation artificialities were 
introduced to the subjects but were probably negligible. 
The subject pool was instruction to prepare their route of 
flight as though it was a day or night flight. In night 
flight map preparation bolder route depictions are used for 
easy visibility in a dark cockpit. 

Additionally, the subject pool was not afforded 
either a flight calculator commonly referred to as a wiz 
wheel or a plotting stencil. Seasoned pilots could easily 
overcome such hurdles. Frankly, they were not included 
because it was felt that although these tools are available 
in fleet map preparation they are not often used and by 
including them might draw attention to their inclusion and 
distract the subjects from preparing as they normally 
would. A protractor was included. 
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b. 


Navigation 


During the task of navigation military helicopter 
pilots use the terrain to mask their position and provide 
cover. Such flight would have them divert from strictly 
adhering to a route of flight. During this experiment the 
subject pool had to overcoming the instinct of flying in 
tactical manner. This proved uncomfortable to many of the 
subjects; many subjects made verbal comments on this point. 
It should be noted that the ability to make dynamic 
tactical modifications to a given route of flight invokes 
and added dimension of complexity to the task of 
navigation. This study wanted to isolate the task of 
navigation itself in the ChrAVE. 

Additionally, the subjects plotting their 
position about every two minutes was not only excessive but 
was also used as a timing device standard by some. 
Plotting was necessary to compile data as to the subject's 
awareness to where they were. Some abandoned their own 
timing and relied on the periodic announcements to estimate 
their timing. 

c. Division of Duties 

Normally there are certain divisions of duties 
amongst the pilots. In order to establish the situational 
awareness of each subject they were instructed to verbally 
report any air traffic and ground activity they may 
encounter. This is fairly normal, but knowing that the PAG 
would not be assisting them may have created undo concerns. 
Furthermore, assistance from the PAG in terrain recognition 
meant that the navigator would have to work harder to 
personally compare the outside view to the map. Normally 
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the pilot at the controls would assist in identifying 
terrain features in front or on the PAC's side of the 
aircraft. The navigator describes what the flying pilot 
should see; the flying pilot confirms the description or 
states what he sees. 

d. Radio Calls 

The task of monitoring the radios varies from 
helicopter community to community, if not from aircrew to 
aircrew. Sometimes the PAG handles the calls, sometimes 
it's the navigator. A novice navigator that is able to 
handle radio calls in addition to the other navigational 
duties is the exception. Most novice navigators are 

usually task limited due to the demands of navigation. In 
this experiment, artificiality was introduced with regard 
to responding to the radios. The subject was told their 
call-sign, "Ugly one-two". Instead of directly responding 
to calls for them, they were instructed to respond by 
saying their "Ugly one-two" as a form of acknowledgement. 

C. DATA COLLECTION 

Data collection came in the form of 

1. Questionnaire & Evaluation Forms (to include 
physiology test results) 

A survey, in the form of a questionnaire (Appendix B), 
was conducted to gather data and information on acceptable 
low-level navigation criteria and prioritized indicators 
effective navigational task performance. Paragraph A.2.c 
of the Methods section has more detailed explanations of 
each of the tests. 

Additionally, physiology tests were conducted on each 
subject with the headgear off and on. 
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a. Visual Acuity 

This test was intended to show the degradation in 
the ability to read writing on the map while hooded. 

b. Color Identification 

This test was intended to show the degradation in 
the ability to correctly perceive colors on the map while 
hooded. 

c. Dvorine Psudo—isochromatic Plates 

This test was intended to determine the extent to 
which the ChrAVE introduces color blindness to subjects 
wearing the HMD. 

d. Hand-eye Coordination Test 

This test was intended to show the degradation in the 
ability to interact physically with the real world while 
hooded. 

Refer to figure 36 to view the results of the 
physiology tests. 

2. Recorded Virtual Flight Data 

The ChrAVE system generated packets containing 
position and orientation data that was written to a file. 
The data was collected about once every second. The data 
was play through a 

3. Maps of the Subjects in the Pool 

The subject pools maps were view following the virtual 
low-level navigation flight. The maps contained the 
periodic plots (position and heading) of the subject as 
well as an indicator of skill used in preparing their map. 
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4. 


Debrief Comments of Subjects in the Pool 


Following low-level navigation flight members of the 
subject pool were debriefed. Many uttered comments that 
were positive both critical. Common comments are noted in 
sections VI-VIII. 

5. Subjects' Peer Evaluations 

In the initial questionnaire questions 20 through 25 
have the subject pool determine acceptable criteria 
limitations for navigational flight. Once each member of 
the subject pool flew the virtual low-level navigation 
route the subject pool was tasked with reviewing the 
performances of each member of the pool and pass judgment 
as to which performances were acceptable and which were 
not. Appendix D contains the peer evaluations from every 
member of the subject pool upon every other member. 
Paragraph A.2.g of the Methods section has more detailed 
explanations of each portion of the peer evaluations and 
the criteria used. 
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VI. 


RESULTS 


Figures 33 and 34 exhibit differences in the 
utilization of preparation tools when land based and sea 
based. Most notable is that satellite photo usage while at 
sea gained eight percentage points over land based usage 
while map study had virtually the opposite effect. This 
may be partly due to the availability of satellite imagery 
while at sea. 


Prep Tools From Shore 


other 

4% 



Figure 33. Preparation tools for low-level terrain 
navigation flights when land based. 
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Prep Tools at Sea 


other 

5% 



Figure 34. Preparation tools for low-level terrain 
navigation flights when land based. 

Attempting to establish the criteria for determining 
effective navigation is difficult. Complicating the issue 
are the numerous circumstances that can occur at any point 
throughout a flight. Two navigational fundamentals are 1) 
knowing your present location and 2) identifying the path 
and checkpoints along intended route. These are 
independent of each other, for example, a navigator may 
know his present location by terrain association but not be 
able to identify points along the intended route. 
Conversely, the navigator may be able to identify points 
along the intended route but estimate his location poorly. 
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other possibilities include a navigator being on course but 
not knowing his location or being unable to identify any 
points along the intended route. This last example 
demonstrates the difficulty in empirically evaluating the 
performance of a navigator; collecting flight data does not 
indicate the navigator's state of mind and situational 
awareness. 

The evaluation slides presented to each subject 
depicted the flight path of a fictitious navigator 
(empirical type data) and the navigator's estimation of 
where the checkpoints were. These estimations give insight 
to the navigator's ability to identify points along the 
intended route. These estimations are not empirically 
collectible in real flight. During actual low-level 
navigation training, the student navigator indicates where 
checkpoints are through conversation and pointing. These 
are not empirically verifiable means, however, it is 
through these means that the navigation instructor gains 
insight to the student navigator's state of mind while 
identifying points along the intended route. 

During the navigational portion of the experiment, 
flight data was empirically collected and the subjects were 
instructed to plot their location periodically. Stress, 
workload, and the degradation in the ability to see and 
interact with the map are all factors led the subject's to 
plot their positions with an unknown degree of error. When 
the subject was told to plot his position, the proctor 
recorded the aircraft's position and heading. Once the 
experiment was complete the proctor created an 8-digit grid 
number and heading for each plot made on the map by the 
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subject. These too contain an unknown albeit manageable 
degree of error. All data was then visually rendered in a 
top-down view. The top-down view allows one to compare the 
actual position and orientation of the subject versus the 
subject's estimation. The differences in heading and 
location are then determined and constitute progression 
towards a useable metric for determining the subjects state 
of mind and situational awareness. See Appendix C for the 
subject pool's top-down. 

The subject pool was anonymously presented the flight 
performances of all other subjects in the pool for 
evaluation. Only the top-down visual renderings were 
presented; they were not privilege to the empirical 
performance data. The peer evaluations that are tallied in 
table 19 (the three right most columns) were based on three 
basic criteria: 

1. The subject's ability to maintain a flight path in 
acceptable proximity to the intended flight path. 

2. The subject's ability to correctly estimate their 
location and orientation. 

3. The subject's overall performance. 

The first two criteria were rated using a scale from 
one to seven, one indicating highly acceptable and seven 
indicating not acceptable. The first two criteria were 
also independent of each other, meaning the proximity to 
the intended flight path was evaluated independently of 
whether or not the subject knew where they were and the 
accuracy of the position estimation was to be evaluated 
independently of whether or not the subjects were on the 
intended route. The last criteria was based on the overall 
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performance of the subjects and evaluated as acceptable 
(pass) or not acceptable (fail). 

Reflecting on the subject pool's collective response 
to question #22, which asked what the threshold between 
acceptable and substandard navigational performance was for 
flight over an intended checkpoint, the subject pool 
collectively stated that flight within 260 meters of a 
checkpoint was acceptable. Based on each subject's flight 
data (Appendix C) the subject pool on average flew within 
260 meters of a checkpoint 3.2 times out of a possible 14 
(see table 19 in Appendix E, center column). 

Granted, some checkpoints were harder to find than 
others. Additionally, the ChrAVE's motion model 
artificially limited the radius of turn to help prevent 
disorientation among the subject's, which meant that 
although subjects may have seen their next checkpoint, they 
may have been limited in their ability to fly over it. 


Proximity to 

a checkpoint 

is 

but 

one 

factor 

in 

identifying effective navigation. 

Question 

#24 

(below) 

of 

the questionnaire 

asked the subject 

pool 

to 

order 

the 


importance of a list of proposed navigational yardsticks. 
It is understood that this is not a complete list of all 
possible criterion that make up effective navigation. 
However, it does attempt to gain insight as to what is most 
important in order to establish metrics for evaluating 
navigational performances. Some of these can be 

empirically evaluated while others appear difficult to 
evaluate without disrupting navigational activity. When 
the subjects were told to plot their position, it was 
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disruptive to their other navigational tasks 
minimally disruptive. 


albeit 


Question 24 of the questionnaire asked the subjects to 
number the following in order of importance: 


Maintaining the route of flight 
Accurately knowing your present location 
(plotting to 8-digit grid accuracy) 

Accurately hitting your checkpoints 
Being off the intended route of flight and 

intending to intercept at the next check 

point 

Knowing your location by reference to a 

dominant terrain feature (plotting to 4-digit 
grid accuracy) 

Seeing your checkpoints, but not hitting them 
Being off the intended route of flight but 

working towards it 


The following table is the result of the above 

question. The subject pool collectively regarded knowing 
one's location by reference to a dominant terrain feature 
(with the ability to plot the position to a 4-digit grid 

accuracy) as most important. This was followed by 

accurately hitting one's checkpoints and then accurately 
knowing one's position to an accuracy of eight digits. 

Next was maintaining the route of flight, followed by the 
off course possibilities. 

Again, determining the extent to which a navigator 
knows their location and the location of the route and 

checkpoints is difficult to empirically ascertain. 



Evaluation Criteria 



Figure 35. Low-level navigation evaluation criteria in 

collective order of importance as indicated by 
the subject pool. 


The results of the physiology tests indicate that 
impairment of prolonged exposure to the ChrAVE lessened 
over time. Most notable is the improvement of hand-eye 
coordination of the subject pool's initial donning of the 
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ChrAVE's headgear to the completion of the flight which on 
average accounted for 63.47 minutes of ChrAVE exposure. 

The physiology results also indicate that the camera 
introduced color distortions that impaired the subjects by 
36.7% of the baseline battery, yet color blindness of any 
sort was not increased by a corresponding amount. 

Lastly, the unhooded post-exposure battery nearly 
replicated the results of the baseline. Although subject's 
appeared to show some lingering effects during the first 
throw of the hand-eye coordination test, rapid improvement 
was noted. Tosses two through ten had nearly identical 
catching results to the baseline. 


Physiology Testing Results 



Exposure Hooded Post- -^ 

Exposure Unhooded 

Post-Exposure 

Figure 36. Physiology results indicating impairment 
during the course of the experiment. 


90 





















Table 2 


is empirical data indicating the closest 
proximity to each checkpoint during the negotiation of the 
On the right of the table there is a column 


route. 
indicating 


each 


proximity to the 


subject's average 
checkpoints. The right most column of the table tallies 
the number of checkpoint proximities that are within the 
subject pool's 260-meter threshold. Only one subject in 

the pool was able to maintain an average distance of less 
than 260 meters. 




Check Poirrts 
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726.2 

755.S 

5385 

11847 

577 

6 

1 

Average Disfance-> 

0.0 

6172 

329.3 

604.6 

1.078,8 

502.0 

587.9 

711.0 

1.045.6 

811 4 

1.453 8 

688 



Difficulty R8nkiog*-> 

N/A 

7 

9 

5 


8 

6 

4 

2 

3 

N/A 




Num Acceptable ***~> 

N/A 

0 

9 

6 

2 

6 

10 

3 

0 

9 

3 

1 




« Note that diffioity ranking was estabC^d pu’ely from ranking tbe average ^stances Addtor^aly. checkpoints 1 and 15 were not 
factored into the checkpoint difficulty ranking because al subjects began at checkpoint 1 and not al subjects were able to complete the 
route, thereby never attempting to navigate to checkpoint 15 

* * Note only checkpoints 2 through 14 were factored into this average because al subjects began at d>eckpoint 1 and not al subjects were 
able to complete the route, thereby never attempting to navigate to checkpoint 15 from the previous checkpoint 
*** Number of checkpoints flown wnhin the subject poors 260 meter tiveshold as established by question #22 of the questionnaire 

Table 2. Subject pool's proximity to each checkpoint. 


Figures 37 and 38 are charts representing possible 
correlation between empirical and subjective performance 
metrics. Both charts have been normalized in scale and 
exhibit each subject's performance according to specific 
metrics. 


Correlation A shows the correlation between the 
subject pool's 260-meter threshold derived from the 
questionnaire before they flew the route and the subject 
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pool's total acceptable performance ratings from the peer 
evaluations after they flew the route. 

Evaluation Correlation A 



001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 

CP Proximitjes < 260 meters ^—Acceptable Overall Performance Ratings 

Figure 37. Evaluation Correlation A. 


Correlation B shows the correlations between 
estimation differences for both heading and position and 
the subjective judgments of the subject pool's peer 
evaluations as well as the subject pool's overall 
acceptance of each subjects performance. 
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Evaluation Correlation B 



001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 


Distance Difference* Heading Difference* Peer Evai Location Score* Acceptabie Overail Performance Ratings 

Figure 38. Evaluation Correlation B. 

The subjects were asked to estimate there percentage 
of time their gaze was spent looking in each sector 
(question 49 of the questionnaire in Appendix B). Figure 
38 depicts estimations of actual flight while figure 39 
depicts estimations while using the ChrAVE. The numbers 
suggest that the ChrAVE is less effective in providing 
information to the user via the chin bubble and far right 
gaze than in the actual flight. There are a few theories 
for this shortcoming. Eirst, the lighting in these areas 
was difficult to smoothly light resulting in shaded 
augmented graphics in the user's view. Second, the 
umbilical of cables coming off of the back of the headgear 
was prohibitive to movement in these sectors. And lastly, 
given only fifteen subjects and their best recollection of 
their gaze habits, these differences are insignificant and 
should be examined in a more scientific manner. 
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Real World Gaze Estimations 


Chin bubble 
6% 


Front 

46% 


Figure 39. Gaze direction estimations while flying actual 
aircraft during low-level land navigation. 



ChrAVE Gaze Estimations 


Right 

11% 



Figure 40. Gaze direction estimations while using the 
ChrAVE for low-level land navigation. 
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VII. CONCLUSIONS 


The overall goal of the ChrAVE prototype system is 
twofold: 

1) To place the subject in an immersive and familiar 
environment, true in first person fidelity with as few 
physically imposed distractions as possible. 

While the ChrAVE was both an immersive and highly 
familiar helicopter setting for the subject pool, it was 
not nearly as true in first person fidelity. The immersive 
nature of the system during task performance suggests that 
mentally first person fidelity may have been fully achieved 
even though physical first person fidelity was generalized 
for any helicopter navigational task. Perhaps true first 
person fidelity can be achieved by appending the ChrAVE 
onto a given pilot's specific type/model/series aircraft. 

2) To exercise the task of navigation as faithfully 
and rigorously as the task is in the real world. 

Observation of the subject pool along with their 
performance and comments suggest that the ChrAVE was 
successful in replicating the task of navigation. That 
said, there are still interface and visual improvements 
that should be made to enhance the experience. 

While motion parallax and coincident EOV is optimized 
shortcomings in the visual presentation remain. Visual 
shortcomings in the present system appear to inhibit or 
delay the user's information gathering ability. This 
disruption makes the user work harder to perform the normal 
task of day VER flight. Improvements made to the visual 
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presentation of the system will likely reduce the workload 
to that expected in actual day VFR flight. 



Figure 41. Coincident fields of view available in SH-60F 
(light gray) and the ChrAVE system (also light 
gray) due to no change in pilots head movement 
behavior. Adapted from (Sikorsky, 1989). 

The average length of time since the last low-level 
navigation flight for the subject pool was nineteen months. 
Skill degradation of the subject pool since their last 
flight is not reportable since no performance data from the 
subject pool's active flying history was available to this 
study; we can only assume the subject pool would have a 
heightened skill set if they were flying on a daily basis. 

In summation, this research, while fertile for follow- 
on focused research from a human interface and task 
performance standpoint, has proved successful. The ChrAVE 
remains a viable system for navigational training and the 
acquisition of spatial knowledge. As such, the ChrAVE 
should continue to be improved and explored. 
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VIII. FUTURE WORK 


It appears that future versions of the ChrAVE have a 
multitude of facets and features to improve on. Because of 
the inherent modularity of the ChrAVE system, improvements 
of one part can be contained within a given module, yet 
benefit the overall product. 

Possible ChrAVE subsystems that lend themselves to 
immediate future work are: 

- Improved headgear assembly and ergonomics . Reducing 
ELD, providing for a more realistic EOV and (variable) 
focal length would result in a more-natural presentation to 
the user. Modeling the headgear to mimic the present 
helmets and NVGs would appear to be the most logical design 
to pursue. 

-Migrate outside VE view and instrument panel 
rendering to a single CPU . As chip speeds increase it may 
be both economically and ergonomically advantageous to 
render the far field VE view and the instrument panel to 
separate windows and channels within the software 
application. However, considering the ultimate goal is to 
append the ChrAVE system onto an existing aircraft, the 
'rendering' of and instrument panel would be replaced with 
producing appropriate signals to drive existing 
implementation. 

-Migrate the chromakey mixing to the video processing 
unit (VPU) . If all camera and VE signals could be mixed on 
a VPU, the need for an external mixing unit and many of the 
signal conversion units would negated, thereby reducing the 
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overall expense of a ChrAVE system. Additionally, the 
reduction in signal conversion throughout the systems 
pipeline means less error propagation in the final 
presentation. 

- Improve chromakey camera reception . Technological 
limitations and recent advancements dictate that there be a 
movement from traditional bluescreen backdrops to a 
LiteRing™ (light emitting diodes (LEDs) mounted around the 
camera lens) and Chromatte™ material (a material 
impregnated with microscopic reflective glass beads). When 
used in concert the LiteRing™ emits chromakey blue light 
that is reflected off of the Chromatte™ material back to 
the lens. This technology does away with the need for 
problematic and costly lighting equipment specific to 
backdrop lighting. Additionally, such technology may 
lend itself to accomplishing dark cockpit, NVG type, 
navigation in the ChrAVE since the camera does not require 
any extra illumination. 

Beyond the ChrAVE system there are a number of 
questions this research stirred up that would be excellent 
fodder for future work. 

- Objective _ performance _ metric _ development . 

Commonsense drove the primitive metrics presented in this 
study. This study embraced the resident expert knowledge 
of the subject pool as well as empirically derived data to 
illustrate a subject's performance relative to the rest of 
the pool. Clearly, more sophisticated data gathering and 
metric development techniques will provide a clearer 
indication of user performance while maintaining 
objectivity. Additionally, it may be possible to gain 
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insight into the user's state of mind as they try to 
navigate. By comparing an intended route of flight to a 
user's virtual route of flight, physical proximity 
performance can be established. But by comparing a user's 
perceived route of flight (a path indicating a navigator's 
perception of where they flew) to his virtual route of 
flight, it is hypothesized that a perceptual proximity 
performance can be established. Comparing the two types 

of proximity may establish why a performance was successful 
or not. Did the navigator know where they were, or were 
they just lucky? If the navigator knew where he was, why 
was he off the intended route of flight? These questions 
may be able to be answered. 

- Networked VEs in collaborative efforts . 

Allowing multiple aircrews to work as a flight could 
benefit the tasks of form and tactical flight, and the 
roles inherent in those tasks. Additionally pilots could 
exercise sound cover and concealment techniques in low- 
level navigation be playing hide & seek. 

-Further viability testing for a ChrAVE deployed 
aboard naval vessel . 

What will be the effects of flying a motionless 
immersive trainer, such as the ChrAVE, on a ship that is 
subject to the movements of the sea? Will a cyber/motion 
sickness emerge or will prolonged exposure force the pilots 
to divorce the seat of the pants feelings from their visual 
perceptions ? 
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Much like its cousin the CAVE, the ChrAVE may prove to 
be a valuable platform from which to launch many human- 
computer interface (HCI) experiments. 

Can a ChrAVE system set in the confines of a cockpit 
be used by a navigating pilot to sufficiently provide that 
pilot with spatial knowledge of an area of flight prior to 
ever having flown there? Clearly there is a difference 
between 'knowing' an area and navigating through or about 
an area. Knowing an area is the product of temporal and 
spatial interactivity exposures, while navigating is the 
process of mental calisthenics during interactivity 
exposures. More specifically, navigation is the method of 
determining position, course, distance passed over, etc. 
Knowing an area means intuitively knowing or recognizing 
your position, relative course and distance to other 
positions. Terrain association is clearly important her, 
but to what degree? 

Vallino's suggests, "the primary performance goal for 
the virtual reality system is to present visual stimuli 
that are consistent with the changes in body position 
sensed by the user." The same can be said for AR, however 
the ChrAVE is not a motion based platform system. Because 
the user is consumed with an inside out view of the virtual 
world while flying the ChrAVE, the user would expect a 
natural connection between the user's internal 
proprioceptive coordinate system and the virtual world 
coordinate system. Considering the goal of this research 
is to ultimately embed a ChrAVE-like system in an aircraft 
embarked on a naval vessel, one can quickly ascertain that 
proprioceptive registration or fusion will be challenging. 
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will this strengthen a pilot's trust 
during instrument flight? Will 

developing skills to consciously 
pants) information from their own 
when relying on instruments to fly? 


in the instrumentation 
this aid pilots in 
divorce (seat of the 
proprioceptive system 
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APPENDIX A. HARDWARE SPECIFICATIONS 


A. V8 HEAD MOUNTED DISPLAY FROM VIRTUAL RESEARCH SYSTEMS 


Display 

• Dual 1.3” diagonal Active Matrix Liquid Crystal Displays 

• Resolution per eye: ((640x3)x480), (921,600 color elements) 
equivalent to 307,200 triads 

• Contrast ratio: 200:1 

Optical 

• Field of view: 60°diagonal 

• Multi-element glass, fully color corrected design 

• Interpupillary distance (IPD) range: 52mm to 74mm 

• Eye relief: Adjustable 10-30mm design accommodates glasses 

• Rubber eye cups prevent eyeglasses and lens contact 

• Overlap: Standard 100% 

Audio 

• Sennheiser HD25 high performance headphones 

• Headphones rotate above headband and snap off when not in use 

Mechanical 

• Single rear ratchet allows for quick, precise fit 

• IPD assembly moves fore/aft to accommodate glasses 

• IPD knobs accessible at sides of shell 

• HMD overall length/width/height: 17.5” x 8” x 6” (43 x 20 x 15 cm) 

• HMD Weight: 34 ounces (1.0 kg) 

Cable 

• Description: Custom molded cable 

• Length 13’ (3.9m) standard 

• Connector: 50 pin SCSI 

Control Box 

• VGA (640 X 480 60Hz) input format 

• Sync on green, separate H and V, or Composite (-i- or - going) 

• Overall brightness and contrast 

• Stereo or mono input auto detected 

• Mono input drives right and left eye with one signal 

• Audio Input: 3.5mm mini stereo phone jack 

• Monitor Output: VGA (640 x 480 60Hz) 

Electrical 

• Power supply: Universal input (-i-5, -i-24, -12, VDC) output 

• Power consumption: 30W 


105 








B. 


GP-US542 3-CCD HIGH PERFORMANCE MICRO HEAD COLOR 
CAMERA WITH DSP FROM PANASONIC 


TV System 

NTSC (Avaiiabie in PAL) 

Pick-up System Micro prism system 

Pick-up Device 

768 (H) X 494(V) 

Three 1/3" interline transfer (IT) supper high sensitivity CCDs 

Scanning 

System 

2:1 Interlace 

525 lines, 60 fields, 30 frames 

Horizontal: 15.734kHz, Vertical: 59.94Hz 

Synchronizing 

System 

Internal or External (Gen-Lock) 

• internai 

NTSC standard (Available in PAL as GP-US532E***) 

• Externai 

(Gen-Lock) 

input 

VBS, VS, HD/VD 

SC Phase for Gen-Lock (VBS): Free adjustable over 360 

H Phase for Gen-Lock (VS): Adjustable 

Video Outputs 

• Video 1,2 

1 .OV [p-p] / 75 ohms NTSC composite video signal, BNC Connector 

• S-ViDEO 
(Y/C) Out 

(Y) 0.714V [p-p] / 75 ohms (C) 0.286V [p-p] / 75 ohms, S-VIDEO 
Connector x 1 

• RGB/SYNC 

(R/G/B) 0.7V [p-p] each / 750 (SYNC) 4V [p-p] / 75 ohms or 0.3V [p-p] 
1750 selectable, D-SUB 9-pin Connector x 1 

Required 

iiiumination 

2000 lx at F8.0 3200K 

Minimum 

iiiumination 

9 lx (0.9 foot candle) at F2.2 with -i-18db gain, 30 IRE level 

Signai-to-Noise 

Ratio 

62dB (Typical, Luminance) without aperture and gamma 

Horizontai 

Resoiution 

750 lines at center (Y signal) 

White Baiance 

ATW (Automatic Tracing White Balance Control), 

AWO (Automatic White Balance Control) and Manual 

Biack Baiance 

ABC (Automatic Black Balance Control) and Manual 

Coior Bar 

SMPTE color bar with 7.5% set-up 

Eiectronic 

Shutter 

ELC (Electrical Light Control) and Manual 

STEP: Selectable 1/60 (OFF), 11100, 1/250,1/500, 1/1000, 1/2000, 
1/4000, and 1/10,000 sec SYNCHRO SCAN: Selectable from 1/525 to 
254/525 line 

Gain Seiection 

AGC, Manual Gain (0, -i-9, -i-18db Selectable) 

Switches 

Power On/Off (POWER), Camera/Color Bar Selection (CAM/BAR), 

Gain UP Selection (OFF/LOW/HIGH (0/-r9/-r18dB), White Balance 
Selection (ATW/AWC/MANU), ELC (Electronic Light Control) On/Off, 
PAGE, ITEM (AWC) <(ABC) and> Scene 1/2 

Controis 

R Gain, B Gain and ELC LEVEL 

Computer 

interface 

RS-232C Control, D-SUB 9-pin Connector x 1 

Lens Mount 

C Mount 
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Power Source 

12V DC 

Power 

Consumption 

8.4 W 

Ambient 

Operating 

Temperature 

32F-113F(0C-45C) 

Ambient 

Operating 

Humidity 

30%-90% 

Dimensions 


• Camera Head 
(Excluding 
Mounting 
Adapter) 

34 (W) X 44 (H) X 52 (D) mm 
[1-5/16" (W) X 1-11/16" (H) X 2" (D)] 

• ecu 

(Excluding 
Rubber Foot 
and connector) 

206.5 (W) X 44 (H) x 250 (D) mm 
[8-1/8" (W) X 1-11/16" (H) X 9-1/2" (D)] 

Weights 

• Camera Head: 

• CCU: 

llOg (0.24 lbs) 

1.7kg (3.74 lbs) 


c. 


LENSES 


1. TV 1,5/4 C From Doctor® 


Type 

Manual Lockable 

Format Sizes 

Up to 

1/3-Inch 

Mount Type 

C 

Focal Length 

f = 4.2mm 

Max. Rel. 

Aperture 

Kmax = 1 .5 

Image Diameter 

2Y’ = 6mm 

Angle of View 

2o = 72 

Number of 
Elements 

7 

Number of 

Groups 

7 

Back Focal 

Length 

s’F’ = 13.5mm 

Front Focal 

Length 

sF = 15.1mm 


DOCTER® TV 1,5/4 C 
TV 1.5/4 CS 



Cumulative Lens 
Thickness 


45.6mm 


Pupil Distances Sep = 15.7mm 
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S’ap = -15.7mm 

Pupil Diameters 

0EP = 2.76mm 

0AP = 19.47mm 

Mount 

C 

CS with adapter 

Mounting Depth 

4.2mm 

Focus Range 

e = 0.15m ...00 

Unit Weight 

.77 / 90g 


2. 12VA6-13 %-inch Format Varifocal Lens from Pelco 


Type 

Varifocal 

Format Sizes 

1/2-inch 

1/3-inch 

Mount Type 

C 

Focal Length 

6-13mm 

Zoom Ratio 

2.2X 

Relative Aperture 
(F) 

1.8-Close 

Operation 

• Iris 

Manual 

• Focus 

Manual 

• Zoom 

Manual 

Angle of View 


• Diagonal 

35.5-75.5 

• Horizontal 

28.5-60.3 

• Vertical 

21.4-45.2 

Minimum Object 
Distance 

0.3m 

Back Focal 

Length 

8.7mm 

Filter Size 

N/A 

Unit Weight 

.20 lb (.09 kg) 


0/W 


I _i I j 

w w 



0/W 

L 

12VA6-13 

1.65 (42.00) 

1.91 (48.60) 


IS-600 MARK 2 MOTION 

TRACKER FROM INTERSENSE™ 

Maximum Angular Rate 

12007sec 

Angular Resolution 

0.02° RMS 

Angular Accuracy 

0.25° RMS 

Maximum Linear Velocity 

15’/sec 

Translation Resolution 

0.01” RMS 
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Translation Accuracy 

0.25” RMS 

Prediction 

0-50ms 

Number of InertiaCube Sensors 

Up to 4 

Number of SoniDisc Beacons 

Up to 8 

Orientation Update Rate 

Up to 500Hz 

Position Update Rate 

Up to 150Hz 

Interface 

RS-232C with selectable baud rates to 115,200 

Protocol 

Compatible with industry-standard protocol 

(FASTRAK™) 


Max. System Configurations GEOS 

PULSAR 

DUAL 

FUSION 

4 

8 

4 6-DOF 

4 6-DOF 

orientation- 

position- 

stations 

stations 

only 

only 



stations 

stations 



Or any combination of Operating Modes that Make 

use of 4 InertiaCubes and 8 SoniDiscs 



Power 

100-240 VAC, 1.0A, 50-60W 


Fusing 

100-120 VAC: T250V, 1.0A 220-240 VAC: T250V, 0.5A 

Operating Temperature 

0to50C (32Fto122F) 


Storage Temperature 

-20to70C (-4Fto158F) 


Dimensions 

Weight 

Cable 

InertialCube orientation sensor 

1.06” X 1.34” X 1.2” 

2.1 oz. 

10’ extendible to 

30’ 

SoniDisc position sensor 

1.0” X 1.0” X 0.65” 

0.4 oz. 

n/a 

X-bar 

41.4” X 3.0” X 1.7” 

8.2 lb. 

20’ extendable to 
34”* 

ReceiverPod (each) 

4.75” X 3.0” X 1.7” 

0.8 lb. 

24” extendable to 
34”* 

Base Unit Signal Processor 

16.75”x 12”x4” 

8.4 lb. 

n/a 

*Total X-bar plus ReceiverPod cable length not recommended to exceed 40’ 

Compatibility 

The lnterSensevlS-600 Mark 2 is compatible with all 
the industry leading software and hardware 

Virtual Research Superscape 

Thomson T&S Softimage 

Kaiser Electro-Optics 

Senses 

Multigen 

Meta VR 
nVision 

Division 

Xtensory 


E. VSC 200D VIDEO SCAN CONVERTER FROM EXTRON ELECTRONICS 
(VGA TO Dl) 


Video Input 


• Number/Signal Type 

1 VGA, 1 Mac RGBHV, RGBS, and RGsB 

• Connectors 

VGA 1 15-pin HD female -i- adapter cable 
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Mac 1 15-pin D female 

• Nominal Level(s) 

Analog 0.7V p-p 

• Minimum / Maximum 
Level(s) 

Analog OV to 2.0V p-p with no offset 

• Impedance 

75 ohms or High Z (switchable) 

• Horizontal Frequency 

Autoscan 24 kHz to 811 kHz 

• Vertical Frequency 

Autoscan 50 Hz to 120 Hz 

• Resolution Range 

Autoscan 560 x 384 to 1280 x 1024 

• External Sync (Genlock) 

0.3V to 1.0V p-p 

Video Processing 


• Encoder 

10 bit digital 

• Digital Sampling 

24 bit, 8 bits per color; 80 MHz 

• Colors 

16.8 million 

• Horizontal Filtering 

4 levels 

• Vertical Filtering 

5 levels 

• Encoder Filtering 

3 levels 

Video Output 


• Number / Type / Format 

1 RGBHV / RGBS / RGsB or component video or 

1 dioital component video (CCIR 6011 / ITU-R 


BT.601)(VSC 200Donlv), or 

1 S-video, or 

1 NTSC / PAL composite video 

• Connectors 

5 BNC female 1 RGBHV / RGBS / RGsB or 

component video 

1 BNC female 1 dioital component video - 

VSC 200D onlv 

1 4-pin mini-DIN female S-video 

1 BNC female composite video 

• Nominal Level 

RGBHV/RGBS/RGsB 0.7V p-p 

S-video and composite 1 .OV p-p 

Impedance 

75 ohms 

Sync 

• Input Type 

Auto detect RGBHV, RGBS, and RGsB 

• Output Type 

RGBHV, RGBS, and RGsB (all RGB formats are swith 
selectable) 

• Genlock connectors 

1 BNC female genlock input 

1 BNC female genlock output (terminate w / 

75 ohms if unsed) 

• Standards 

NTSC 3.58 and PAL 

• Input Level 

1.5V to 5.0V p-p 

• Output Level 

5V p-p 

• Input Impedance 

75 ohms 
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Output Impedance 
Polarity 


75 ohms 
Negative 


F. ADC-6801 SIGNAL CONVERTER FROM LEITCH (RGB TO Dl) 


Input 


• Sampling Rate 

27MHz Y 13.5MHz Cr/Cb 

• Quantization 

10 bits 

• Input Standards 

SMPTE / EBU, Mil, Betacam component or RGB at 525 
or 625 lines rates 

• 5BNCs 

Ext. Sync, Loop Through G/Y, B/B-Y, R/R-Y 

Component Analog Input 


• Connector 

BNC per lEC 169-8 

• Impedance 

75 ohms unbalanced 

• Signal Level 

1 V 

• Adjustable Gain 

±10% 

• Time Adjustment Range 

±1.8ps 

• Return Loss 

>40dB to 5.5 MHz 

Filtering As Per CCIR 601 
Specifications 


• Frequency Response 

Y channel ±0.1 dB to 5.5 MHz 


Cr, Cb Channels ±0.2 dB to 2.75 MHz 

• Signal to Noise Ratio on 
all Channels 

>64 dB RMS, relative to 0.714 V, 10 kHz to 5.5 MHz 

• Interchannel Crosstalk 

<-50dB 

• 2T K factor 

<0.5% 

• Luminance Non-linearity 

<1% 

• Gain Alignment 

<1%, typically better than 0.5% 

• DC Clamping 

Typically within 1 quantization level on field average. 


Output 

• Output Standard 4:2:2, two BNCs as per SMPTE 259 

• Input to Output Delay 3.6ps 

G. ULTIMATTE 400-DELUX COMPOSITE VIDEO MIXER FROM THE 
ULTIMATTE CORPORATION 

Specifications • Conforms to CCIR 601 

• 10-bit or 8-bit SDI inputs and outputs 

• Internal Foreground and Matte processing 4:4:4:4 

• 525 / 625 Auto-selectable 
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Video 


• I/O Resolution 

4:2:2 

• FG Input 

4:2:2 

• BG Input 

4:2:2 

• Matte In 

4:0:0 

• Digital Reference 

4:2:2 

• FG and BG Out 

4:2:2 

• Internal FG Processing 
and Matte Generation 

4:4:4:4 

• Inputs 

Serial CCIR 601, BNC 75 

• Outputs 

Serial CCIR 601, BNC 75 

SDC-100 SERIAL DIGITAL TO VGA MONITORING CONVERTER 
FROM LEITCH (Dl TO VGA) 

Serial Digital Input 

BNC 75 ohm; 270Mb/s; 259M-C 

Up to 100m automatic cable equalization 

Input Return Loss 

13.9 dB at 270 MHz 

VGA Monitor Output 

Sub-D 15-pin female connector 


RGB ±3dB0.7V, H+VTTL 

Frequency Response 

• Luminance 

• Chrominance 

• Gamma Correction 

• Standards 

• Signai-to-Noise 

625 iine / 50 Hz mode with 
iine doubiing 

• Horizontai Frequency 31.25 kHz 

• Verticai Frequency 50 Hz 


525 line / 60 Hz mode with 
line doubling 


• Horizontal Frequency 

31.469 kHz 

• Vertical Frequency 

59.94 Hz 


I. VE CPU 

Manufacturer / Modei Dell / Dimension 8100 


±0.5 dB from DC to 5.25 MHz 
±3 dB up to 10 MHz 

±3 dB up to 4 MHz 

Automatic 

525-line and 625-line auto switching 
-64 dB 
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CPU 


Intel® Pentium® 4 
1300 MHz 


Memory 

128 MB RAM 

Operating System 

Microsoft Windows 2000 

5.00.2195 

Service Pack 2 

Monitor 

Set to 640 X 480 for HMD compatibiiity 

60 Hz 

Power 

Industry Standard for U.S. desktop computers 


INSTRUMENT PANEL CPU 

Manufacturer / Model 

SGI / Silicon Graphics 320/540 

CPU 

X86 Family 6 Model 7 Stepping 2 
SGI-320_ARCx86_mp 

Memory 

200 MB RAM 

Operating System 

Microsoft Windows 2000 

5.00.2195 

Service Pack 2 

Monitor 

SGI 1600 SW 

60 Hz 

Display Adapter Information 
• Graphics Processor 

GeForce2 MX/MX 400 

• Bus Type 

AGP 

• Bios Version 

3.11.01.17.20 

• On-Board Memory 

32 MB 

• TV Encoder Type 

Conexant Bt869 

Power 

Industry Standard for U.S. desktop computers 

TOP DOWN (PLOTTER) VIEW CPU 

Manufacturer / Model 

Dell / Dimension 4100 

CPU 

Intel® Pentium® 4 

1300 MHz 

Memory 

128 MB RAM 

Operating System 

Microsoft Windows 2000 

5.00.2195 

Service Pack 2 

Monitor 

Set to 1024 X 786 

60 Hz 

Power 

Industry Standard for U.S. desktop computers 
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APPENDIX B. QUESTIONNAIRE & EVALUATION SHEET 

ChrAVE Experiment & Questionnaire 

Please read first '. The following experiment and questionnaire are completely 
confidential. Nothing you do or answer will be related back to you in any manner. Thank 
you for your assistance. Please begin below the solid line and hand to the proctor when 
you reach “Stop Here”. You may ask questions at any time. 

Subject Number_(proctor use only) 

Preliminary questions: 

1. Do you have any history of epilepsy? Yes / No 

2. Are you prone to simulator sickness? Yes / No 

3. Do you require corrective lenses? Yes / No 

4. What is your vision uncorrected? 

5. Do you have any other history of eye disease or injury? 

6. How often do you use a computer on a daily basis? (Check one.) 

O 0-2 hours O 2-4 hours O 4-6 hours O 6-8 hours O greater than 8 hours 

7. Have you ever used virtual environment for training or entertainment? Yes / No 

8. If yes, did you use a head-moimted display (HMD)? Yes / No 

9. As a designated aviator, how would you rate your low level helicopter navigational 
skills? (Check one.) 

O novice O average O advanced O navigation instructor O expert 

10. List all type, model, series aircraft you are or have you been qualified to fly. 

(Disregard flight school aircraft unless you were a flight school instructor.) 



11. About how many hours of flight time do you currently have? _ 

12. About how many hours of night vision goggle (NVG) flight time do you currently 

have? _ 

13. When was your last low level helicopter navigation map preparation? 

Month_ Year_ 
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ChrAVE Experiment & Questionnaire 

20. Are you familiar with Fort Irwin area depicted on the map issued to you for this 
experiment? Yes / No 

If yes: 

Have you ever flown in the area? Yes / No 

The following questions ask your opinion of acceptable criteria for low-level navigation 
as it pertains to the terrain of the Fort Irwin area during non-tactical flight. You may 
refer to you map at any time. 

21. Being within_meters of the intended route of flight is the threshold for 

acceptable and substandard navigational performance. 

OlOO 0200 0300 0400 0500 0600 0700 0800 0900 OlOOO OMore 

22. Being within_meters of the intended checkpoints is the threshold for acceptable 

and substandard navigational performance. 

OlOO 0200 0300 0400 0500 0600 0700 0800 0900 OlOOO OMore 

23. During flight between checkpoints it is acceptable for the route of flight accuracy 
threshold to decrease. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

24. Number the following in order of importance: 

_Maintaining the route of flight 

_Accurately knowing your present location (plotting to 8 digit grid accuracy) 

_Accurately hitting your checkpoints 

_Being off the intended route of flight and intending to intercept at the next 

check point 

_Knowing your location by reference to a dominant terrain feature (plotting to 

4 digit grid accuracy) 

_Seeing your checkpoints, but not hitting them 

_Being off the intended route of flight but working towards it. 

25. Evaluation task: 

You will be provided numerous map slides of the Fort Irwin area. The green path is 
the intended route of flight. Green circles are intended checkpoints. The black or red 
path is a recorded navigational performance. The black or red path is the where the 
subject flew. The black or red circles indicate where the subject identified the 
checkpoint. (Note that subjects not on the intended route of flight can still correctly 
identify a checkpoint and that subjects on the intended route of flight can mis-identify 
a checkpoint.) It is your task to separate these slides into acceptable and non- 
acceptable performance. 
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Figure 42 


Evaluation Slide B. 


117 


























































































Figure 43 


Evaluation Slide D. 
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Figure 44 


Evaluation Slide F. 
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Figure 45. 


Evaluation 


Slide G. 
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Figure 47 


Evaluation Slide K. 
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Figure 48 


Evaluation Slide M. 
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Figure 49. 


Evaluation Slide N. 
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Figure 50. 


Evaluation Slide P. 
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Figure 51. 


Evaluation 


Slide Q. 
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Figure 52 


Evaluation Slide R. 
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Figure 53. 


Evaluation Slide T. 
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Figure 54 


Evaluation Slide U. 
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Figure 55 


Evaluation Slide W. 
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Figure 56. 


Evaluation Slide Y. 
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ChrAVE Experiment & Questionnaire 


The Goal: 

To become sufficiently familiar with the terrain in the Ft Irwin area so as to successfully 
navigate as much of the route of flight as possible in thirty minutes by providing soxmd 
voice commands while using the Chromakeyed Augmented Virtual Environment 
(ChrAVE) system. 


Your Resources: 

-2 satellite images 

- an image of the Fort Irwin area 

- an image concentrated on and depicting the general route of flight 
-A 1:50,000 map of the Ft Irwin area (you may write on the map) 

-A kneeboard route card (you may write on the route cards) 

-A set of map pens 

-A collection of rulers, protractors, and stencils 

-Two knee boards to use during the flying portion of the experiment 

-A clock for timing (you may want to become familiar with its operation) 

-One pair of flight gloves required during flight 


The Tasks: 

Prepare the man : Utilizing the information on the kneeboard route cards and the other 
resources prepare the 1:50,000 map. You will be relying on this map during the virtual 
flight portion of the experiment. T^is will be a non-tactical flight; emphasis is on terrain 
association and navigation. Prepare your map as though you were unsure whether your 
flight would be conducted during the day or at night. Take note of key terrain such as 
checking features, charmeling features, and limiting features. When you are comfortable 
with your preparation inform the proctor. 

Maintain awareness of vour location : You will be required to plot your direction and 
position on the map on demand (approximately every two minutes). Place an arrow ('t') 
to represent your direction; the point of the arrow shall represent your position. The 
proctor will call out a number for you to place by the arrow. You will be able to refer to 
the map, instrument panel, the timing clock, and the virtual world during the flight. 
Remember your view to the left will be limited to the edge of the blue screen. It will be 
important to associate terrain on the map with the terrain in the virtual world in order to 
maintain your position. Some (not all) roads are identifiable in the virtual environment. 
As a rule of thumb, roads and other manmade features clearly identifiable in the satellite 
imagery are identifiable in the virtual enviromnent. If you are lost, you may instruct the 
pilot at the controls to orbit in order to regain your orientation. However, your goal is to 
navigate as much of the route as possible in the time allotted (thirty minutes). 

Navigation : Successfully navigate as much of the route of flight as possible in thirty 
minutes. 


4 Subject Number: 
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Figure 57. Satellite Image of Fort Irwin Area with 
intended route of flight overlaid. 
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ChrAVE Experiment & Questionnaire 

Monitor the radios : There will be radio chatter in the Fort Irwin area. Your aircraft’s 
callsign is “Ugly one-two”. You are required to answer radio calls only to vour aircraft 
by pressing the button on the cyclic and saying “Ugly one-two, roger.” Disregard all 
other chatter. You may ask the proctor to adjust the volume for you. 

Direct the pilot at the controls : During the flight portion of the experiment you will direct 
the flight of your aircraft by giving appropriate voice commands to the pilot at the 
controls (the proctor). Directional voice commands are restricted to; 

“ Left turn ” / “ Right turn ” - These commands start a standard rate turn. 

“ Easy left turn ” / “ Easy right turn ” - These commands start a half standard rate turn. 
“ Stop turn ” - This command levels the wings. 

O’clock position calls - These commands start a standard rate turn followed by and 
automatic rollout. Turns to 6 o’clock will be right hand turns unless “ Turn left to 6 
o’clock ” is requested. A turn to one o’clock means a heading change of 30 degrees, two 
o’clock means a heading change of 60 degrees, etc. 

“ Orbit left ” / “ Orbit right ” - These command should be used only when attempting to 
reestablish your orientation. These commands will initiate a standard rate turn that 
maintains altitude (climbs will be initiated only to avoid terrain). Remember a full turn 
of 360 degrees will take two minutes. 

If there is any type of problem, the terms “ Game over ” or “ Pause game ” will pause the 
game. 

Maintain situational awareness : There will be other activity in the Fort Irwin area while 
you are navigating. Simply call the traffic or activity to the rest of your flight crew when 
you see it. 

Cockpit management : Practice effective cockpit management skills. Remember this 
trainer will be imfamiliar to you. Plan your map folding adjustments during long straight 
stretches of flight. Organize necessary resources within your reach. 

Flight parameters: 

Your aircraft is a ‘generic’ helicopter. 

You will be flying at 90 knots on a windless day. Airspeed equals groundspeed. 

-In one minute of flight at 90 knots your aircraft will travel 1.5 NM or 2.778 KM. 

-In two minutes of flight at 90 knots your aircraft will travel 3.0 NM or 5.556 KM. 

The pilot at the controls will ‘visually’ maintain about 200 feet AGL. However if 
orbiting he will maintain 200 feet above the highest object in the orbit path. 

Remember to lead your roll out calls; it takes longer to roll out from a standard rate turn 
than a half standard rate turn. 

Do you have any questions? 


5 Subject Number: 
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ChrAVE Experiment & Questionnaire 

Ensure the subject knows how to use the clock. 

Check for history of epilepsy or proneness to simulator sickness 

Unhooded Tests: 

Eye Test: 

Line:_Number correct:_ 

Hand/Eye Coordination Test: 

1 throw: Caught / Fumbled / Dropped 

2"'’ throw: Caught / Fumbled / Dropped 
3"* throw: Caught / Fumbled / Dropped 

4*'’ throw: Caught / Fumbled / Dropped 

5*'’ throw: Caught / Fumbled / Dropped 

6*'’ throw: Caught / Fumbled / Dropped 

7*'’ throw: Caught / Fumbled / Dropped 

8'*’ throw: Caught / Fumbled / Dropped 

9* throw: Caught / Fumbled / Dropped 

10* throw: Caught / Fumbled / Dropped 


Hooded Tests: 

Subject was hooded at time 

Provide instructions on HMD adjustments and a warning on twisting the inertial tracker. 
Eye Test: 

Line:_Number correct:_ 

Hand/Eye Coordination Test: 

1 throw: Caught / Fumbled / Dropped 

2"“* throw: Caught / Fumbled / Dropped 

3"^ throw: Caught / Fumbled / Dropped 

4*'’ throw: Caught / Fumbled / Dropped 

s'*' throw: Caught / Fumbled / Dropped 

6*'' throw: Caught / Fumbled / Dropped 

7''’ throw: Caught / Fxunbled / Dropped 

8* throw; Caught / Fumbled / Dropped 
9* throw: Caught / Fumbled / Dropped 

10* throw: Caught / Fumbled / Dropped 


6 Subject Number: 


Color Identification Test: 
Blue: Pass / Fail 

Red: Pass / Fail 

Green: Pass / Fail 

Orange; Pass / Fail 

Purple: Pass / Fail 

Black: Pass / Fail 

Dvorine Plates Test; 

48 67 38 92 70 
95 26 2 74 62 


Color Identification Test: 
Blue: Pass / Fail 

Red: Pass / Fail 

Green; Pass / Fail 

Orange: Pass / Fail 

Purple: Pass / Fail 

Black; Pass/Fail 

Dvorine Plates Test; 

48 67 38 92 70 
95 26 2 74 62 
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ChrAVE Experiment & Questionnaire 

(proctor use only) 

Flight Portion: 

Set up: 

1. Set-up ChrAVE. 

2. Prep sound file to minute ten. 

3. Start Vega Desert Trap scenario on ChrAVE cart. 

4. Start Vega Instrument Panel on the SGI. 

5. Start Top View on the desktop PC. 

6. Set stopwatch to 3 minutes. 

7. Configure Desert Trap scenario: 

a. Turn off flight paths (buttons ‘Z’, ‘X’, and ‘C’) 

b. Press ‘A’ to place subject at the alternate practice position. 

c. Ensure the compass matches the users answer to question 16. Magnetic / Grid 

8. When the subject is ready press ‘P’ to toggle off pause. 

9. Start sound file. 

10. Have subject start the stopwatch to begin his prep-time. 

11. Prep sound file to beginning. 

12. Press “R” to place the subject at the start position. 

13. When the subject is ready press ‘P’ to toggle off pause. 

14. Start sound file. 

Radio Calls: 

1“ call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
2"'' call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
3'*' call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
4* call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
S"* call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
6"’ call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
T"" call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
s'* call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
9"’ call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
1O* call: OCorrectly heard & acknowledged 

ONot acknowledged 
OAcknowledged call for others 
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Other comments: 

Head movements: Subtle / Moderate / Rapid 
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ChrAVE Experiment & Questionnaire 


Hooded Tests: 


Eye Test: 



Line: 

Number correct: 


Hand/Eye Coordination Test: 

Color Identification Test: 

L' throw: 

Caught / Fumbled / Dropped 

Blue: Pass / Fail 

2"'* throw: 

Caught / Fumbled / Dropped 

Red: Pass / Fail 

3'''* throw: 

Caught / Fumbled / Dropped 

Green: Pass / Fail 

4'*’ throw: 

Caught / Fumbled / Dropped 

Orange: Pass / Fail 

5* throw: 

Caught / Fumbled / Dropped 

Purple: Pass / Fail 

6* throw: 

Caught / Fumbled / Dropped 

Black: Pass / Fail 

7* throw: 

Caught / Fumbled / Dropped 


8* throw: 

Caught / Fumbled / Dropped 

Dvorine Plates Test: 

9* throw: 

Caught / Fumbled / Dropped 

48 67 38 92 70 

lO* throw: 

Caught / Fumbled / Dropped 

95 26 2 74 62 


Unhooded Tests: 


Subject was unhooded at time 


Total HMD exposure: 


Eye Test: 



Line: 

Number correct: 


Hand/Eye Coordination Test: 

Color Identification Test: 

throw: 

Caught / Fumbled / Dropped 

Blue: Pass / Fail 

2"'* throw: 

Caught / Fumbled / Dropped 

Red: Pass / Fail 

3'^'’ throw: 

Caught / Fumbled / Dropped 

Green: Pass / Fail 

4* throw: 

Caught / Fumbled / Dropped 

Orange: Pass / Fail 

5*’' throw: 

Caught / Fumbled / Dropped 

Purple: Pass / Fail 

6* throw: 

Caught / Fumbled / Dropped 

Black: Pass / Fail 

7* throw: 

Caught / Fumbled / Dropped 


8* throw: 

Caught / Fumbled / Dropped 

Dvorine Plates Test: 

P* throw: 

Caught / Fumbled / Dropped 

48 67 38 92 70 

10* throw: 

Caught / Fumbled / Dropped 

95 26 2 74 62 

Ask the subject if he/she had to adjust to the real world when the HMD came off? 

Yes / No 
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ChrAVE Experiment & Questionnaire 


Post-flight Questions: 

ChrAVE Activity 

26. Did you observe the enemy armor unit? Yes / No 
If yes: 

What color smoke was coming from the enemy armor unit?_ 

How many tanks did you observe? OZero OOne OTwo OThree OFour 

How many personnel carriers did you observe? OZero OOne OTwo OThree OFour 

27. Did you observe the downed aircraft? Yes / No 
If yes: 

What color smoke was coming from the aircraft? _ 

How many aircrew did you observe? OZero OOne OTwo OThree OFour 

28. Did you observe the small town in the vicinity of Nelson Airfield? Yes / No 
If yes: 

Did you could observe any vehicle traffic in the small town? Yes / No 
Did you could observe the church in the small town? Yes / No 

29. Did you observe any evidence of close air support (CAS) traffic? Yes / No 
If yes: 

What did you observe? OHarrier(s) OContrails from a Jet OCanopy reflection / flash 

30. Did you observe any other aircraft? Yes / No 
If yes: 

What aircraft did you see? (Check all that apply) 

OUH-l OCH46 OCH-53 OV-22 ODon’tknow 

ChrAVE Performance 

(Select the appropriate response following each statement.) 

31. Navigating in the ChrAVE resembled the actual task of navigation? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

32. The voice commands I gave in the ChrAVE resembled actual voice commands used 
in terrain flight navigation? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 
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ChrAVE Experiment & Questionnaire 

33. The ChrAVE performs as well as visual simulators I have used in the past with regard 
to terrain flight navigation? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

34. The ChrAVE is more valuable as a tool than desktop simulators I have used in the 
past with regard to terrain flight navigation? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

35. The ChrAVE required me to use cockpit management skills similar to cockpit 
management skills I use in actual aircraft? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

36. If the ChrAVE was able to transform aircraft aboard underway ships into the 
navigational tool I saw today, I would be likely to use it? 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

37. Viewing the map through the HMD was acceptable. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

38. Viewing the kneeboard card on the kneeboard through the HMD was acceptable. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

39. Viewing the instrument panel through the HMD was acceptable. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

40. The terrain appeared realistically in size and dimension. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 


ChrAVE Aftereffects 

41. The ChrAVE made me feel queasy / nauseous. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 


42. The ChrAVE is disorienting because it is a motionless platform. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

43. The ChrAVE provides a 60-degree field of view (FOV). If a wider field of view 
were available it would be more beneficial. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

44. If a wider field of view were available it would induce less discomfort/nausea. 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

45.1 would want a wider FOV even if the resulting headgear were heavier^ulkier? 
OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 
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ChrAVE Experiment & Questionnaire 

Experiment Tasks 

46. The tasks were realistic. 

Preparing the map 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Maintaining awareness of your location 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Terrain association 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Navigation 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Monitoring radio calls 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Directing the pilot at the controls / Voice commands 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Maintaining situational awareness / seeing other aircraft or activity 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

Cockpit management skills 

OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

47. The workload in the ChrAVE was the same as it is in real world low-level helicopter 
navigation. 

a. OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

b. OLess OSanie OMore 

48. The stress level in the ChrAVE was the same as it is in real world low-level 
helicopter navigation. 

a. OStrongly agree OAgree ONeither agree nor disagree ODisagree OStrongly disagree 

b. OLess OSame OMore 


49. Enter the percent of time your gaze is/was directed in each of the following glassed 
regions. 
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ChrAVE Experiment & Questionnaire 


50. What suggestions for improvements of the ChrAVE do you have? Please add any 
other statements you may have concerning this experiment. (If you have a comment 
on a specific question please provide the question number.): 



Thank you for your participation. 
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APPENDIX C. SUBJECT FLIGHT DATA 
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Table 3 


Subject pool's proximity to each checkpoint 


subjects were able to complete the route, thereby never attempting to navigate to checkpoint 15. 

Note only checkpoints 2 and 14 were factored into this average because all subjects began at checkpoint 1 and not all subjects were able 
to complete the route, thereby never attempting to navigate to checkpoint 15 from the previous checkpoint. 
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Table 4. SubjectOOl's Location, 

Proximity Results. 
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Figure 58 


SubjectOOl's Flight Path and Position Plots 
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Table 5. Subject002's Location, Direction, & 

Proximity Results. 
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Figure 59. Subject002's Flight Path and Position Plots. 


147 


Estimated Position 

















& Proximity Results 

Check Points 

Min Distance 

from Flown 

Path to 

Checkpoint 

(Meters) 

o 

o 

271.0 

311.2 

(O 

8ZS€ 

882.3 

336 

357.8 


33J7.1 

3.2202 

O 

? 


Northing 

3130 

2750 

2030 

167 0 

167 0 

161 0 

202 0 

264 0 

293 0 

2780 

3130 

A 

i 

<9 

? 

Easting 

490 0 

483 0 

4080 

360 0 

0816 

o 

fO 

« 

204 0 

235 0 

3080 

437.0 

490.0 

< 

Check Point Name 

Course Entry Point 

Cemetery Bend 

Granite Pass 

Whiskey Saddle 

Romeo HiB 

Hill 1161 

Nelson Lake 

McLean Lake Feed 

Tri-valley Pass 

Drinkwater Lake 

Course Entry Point 


- 

or 

m 

- 

in 


- 

«> 




r-- 
















Directiol 

Difference 

Heading 

(Deg) 


20.1 

22 





o 

533 

22.3 

991 

7 1 

119 1 

389 

44 5 

32.9 


Distance 

(Meters) 


ZZE9 

20606 





1010.6 

(n 

§ 

1.494 7 

5125 

2.790 3 

2291.9 

60878 

6B97.0 

2543.6 

Subject003’s Location, 

Subiect‘8 Map Plots Subiecl's VE Positions 

Heading 

(Gnd) 


2439 

197 9 





277 0 

266 7 

rs, 

CTk 

rn 

0166 


095 9 

078 9 

in 

IN 

s 

Averages--> 

anks indicate subiect did not plot 

Northing 


204 5 

171 8 





174 8 

1381 

178 2 

2209 

245 8 

249 1 

244 0 

oo 

% 

Eastmg 


4181 

394 9 





306 0 

265 5 

oo 

*“4 

201 8 

243 2 

246 6 

300 3 


Heading 

(Grid) 


2640 

2000 





2720 

320 0 


3600 

050 0 

2150 

040 0 

OSH] 

Northing 


201 5 

167 5 





1655 

1550 


225 5 

2660 

256 0 

247 1 

w 

n 

Easting 


4125 

374 7 





3020 

258 0 


204 1 

224 0 


239 5 

q 

S 


Time 

o 

i 

fN 

i 

(JO 

8 

s 

8 

1 

8 

R 

8 

o 

S 

S 

8 


8 

8 

s 

8 

5 

8 

IS 

r,: 



OI 

€0 


i 

iD 


00 

<T> 



(N 





CQ 

• 

_ 




Table 6. SubjectOOS's Location, Direction, & 

Proximity Results. 
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Figure 60. 


SubjectDOS's Flight Path and Position Plots. 
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Table 7. Subject004's Location, Direction, & 

Proximity Results. 
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Figure 61 


Subject004's Flight Path and Position Plots 
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Recorded Flight Path 
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Estimated Position Icons 










Table 8. SubjectOOS's Location, Direction, & 

Proximity Results. 
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Figure 62. SubjectOOS's Flight Path and Position Plots. 
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Table 9. Subject006's Location, Direction, & 

Proximity Results. 
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Figure 63. Subject006's Flight Path and Position Plots. 
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Table 10. SubjectOOV's Location, Direction, & 
Proximity Results. 
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Figure 64. 


SubjectOOV's Flight Path and Position Plots. 
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Estimated Position 
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Table 11. SubjectOOS's Location, Direction, & 
Proximity Results. 
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Figure 65. SubjectOOS's Flight Path and Position Plots. 
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Estimated Position 












Table 12. Subject009's Location, Direction, & 
Proximity Results. 
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Figure 66. Subject009's Flight Path and Position Plots. 
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Estimated Position 










Table 13. SubjectOOS's Location, Direction, & 
Proximity Results. 
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Figure 67. SubjectOlO's Flight Path and Position Plots. 
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Table 14. SubjectOll's Location, Direction, & 
Proximity Results. 
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Figure 68. SubjectOil's Flight Path and Position Plots. 
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Table 15. Subject012's Location, Direction, 
Proximity Results. 
























































Figure 69. Subject012's Flight Path and Position Plots. 
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Table 16. SubjectOlS's Location, Direction, & 
Proximity Results. 
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Figure 70. SubjectOlS's Flight Path and Position Plots. 
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Estimated Position 








Table 17. Subject014's Location, Direction, & 
Proximity Results. 
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Figure 71. Subject014's Flight Path and Position Plots. 
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Table 18. SubjectOlS's Location, 
Proximity Results. 
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Figure 72. SubjectOlS's Flight Path and Position Plots. 
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Estimated Position 






A survey, in the form of a questionnaire (Appendix A), 
was conducted to gather data and information on acceptable 
low-level navigation criteria and priorities. 

Additionally, physiology tests were conducted on each 
subject with the headgear off and on. 

1. Visual acuity-intended to show the degradation in 
the ability to read writing on the map while hooded. 

2. Color identification-intended to show the 
degradation in the ability to correctly perceive colors on 
the map while hooded. 

3. Dvorine psudo-isochromatic plates-intended to 
determine the extent to which the ChrAVE introduces color 
blindness to subjects wearing the HMD. 

4. Hand-eye coordination test-intended to show the 
degradation in the ability to interact physically with the 
real world while hooded. 

Refer to figure 36 to view the results of the 
physiology tests. 
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APPENDIX D. PEER EVALUATIONS 



Location, Direction, & Proximity Result 
Averages 

Peer Evaluations 

Subject 

Position & 
Orientation 
Differences 

Checkpoint Proximities 

Distance 

(Meters) 

Heading 

(Deg) 

Min 

Distance 
from Flown 
Path to 
Checkpoint 
(Meters) 

Number of 
Acceptable 
Checkpoint 
Proximities 
*4 

Average 
Proximity 
of Paths 
Score 
*1 

Average 

Location 

Estimation 

Score 

*2 

Total 

Acceptable 

Cverall 

Performance 

Ratings 

*3 

001 

1,124.9 

14.4 

378.1 

4 

2.4 

1.9 

14 

002 

965.5 

21.7 

326.7 

5 

1.8 

2.1 

14 

003 

2,543.6 

32.9 

1346.0 

1 

6.3 

5.5 

0 

004 

2,403.2 

30.0 

1279.9 

4 

4.9 

4.9 

5 

005 

1,521.1 

26.8 

924.1 

4 

3.8 

3.5 

12 

006 

4,586.1 

28.1 

1416.3 

1 

6.6 

6.4 

0 

007 

1,435.5 

17.1 

739.3 

3 

3.5 

3.4 

13 

008 

1,872.5 

18.6 

575.2 

3 

3.4 

3.6 

14 

009 

1,132.1 

12.1 

533.8 

4 

2.7 

2.5 

14 

010 

1,054.9: 

21.7 

231.9 

5 

1.8 

2.2 

14 

oil 

1,677.9 

19.2 

473.1 

4 

2.3 

2.8 

14 

012 

1,133.9 

19.4 

558.4 

4 

3.7 

3.1 

14 

013 

1,587.6 

30.7 

576.9 

2 

4.7 

3.8 

7 

014 

880.1 

7.1 

481.2 

3 

2.6 

2.1 

14 

015 

1,172.21 

18.2 

579.4 

1 

4.1 

3.5 

10 

Averages-> 

1,672.7 

21.2 

694.7 

3.2 

3.6 

3.4 

10.6 


*l. The subject's ability to maintain a flight path in acceptable proximity to the intended path . 

Rated using a 1 to 7 scale,indicating highly acceptable while 'T indicates 
not acceptabie. 

♦This criteria is independent of the foiiowing criteria, meaning the proximity to the intended 
flight path was evaiuated independents of whether or not the subject knew where they were. 

*2. The subject's abiiitv to correctiv estimate their iocation . 

Rated using a 1 to 7 scale, T' indicating highly acceptable while 7' indicates not acceptable. 

♦This criteria is independent of the preceding criteria, meaning the accuracy of the position 
estimation was to be evaluated independently of whether or not the subjects were on the 
intended route. 

*3. The overall performance . 

Rated the overall performance as acceptable or not acceptable ('A' or'N') 

*4. Number of checkpoints flown within the subject pool's 260 meter threshold as established 

by question #22 of the questionnaire. 

Table 19. Summary of subject pools empirical data and 
peer evaluations. 
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PROXIMITY OF PATHS EVALUATIONS 



Subject Number 

001 

002 

003 

004 

005 

006 

007 

008 

009 

010 

Oil 

012 

013 

014 

015 

Average 

Grade 

Harshness 

Ranking 


Subject 001 


2 

6 

5 

4 

7 

3 

3 

3 

2 

2 

3 

4 

3 

4 

3642857 

7 


Subject 002 

3 


7 

7 

5 

7 

4 

4 

3 

2 

3 

4 

6 

3 

4 

4,428571 

2 


Subject 003 

3 

3 


4 

4 

6 

4 

3 

3 

3 

3 

4 

4 

3 

4 

3642857 

7 


Subject 004 

2 

2 

6 


4 

6 

4 

3 

2 

1 

2 

2 

4 

2 

3 

3.071429 

12 

q: 

Subject 005 

2 

1 

5 

4 


7 

2 

3 

2 

1 

1 

3 

4 

2 

4 

2.928571 

13 

Subject 006 

2 

1 

7 

4 

3 


3 

2 

3 

2 

2 

3 

4 

2 

3 

2.928571 

13 


Subject 007 

3 

2 

6 

5 

4 

7 


5 

3 

2 

3 

5 

6 

4 

5 

4 285714 

3 

< 

Subject 008 

1 

1 

7 

4 

3 

7 

4 


3 

1 

1 

4 

5 

2 

4 

3.357143 

10 

_i 

<r 

Subject 009 

3 

3 

7 

6 

5 

7 

5 

5 


2 

2 

5 

6 

2 

6 

4.571429 

1 

s 

Subject 010 

3 

1 

6 

3 

3 

6 

3 

3 

2 


3 

3 

5 

2 

4 

3.357143 

10 


Subject Oil 

3 

3 

6 

4 

4 

7 

3 

3 

3 

2 


4 

5 

2 

5 

3.857143 

6 


Subject 012 

1 

1 

5 

5 

2 

6 

2 

3 

2 

1 

2 


4 

3 

4 

2.928571 

13 


Subject 013 

3 

2 

7 

6 

4 

7 

4 

3 

3 

2 

3 

4 

3 

4 

3.928571 

5 


Subject 014 

3 

2 

7 

6 

4 

7 

4 

4 

3 

2 

2 

4 

5 


4 

4.071429 

4 


Subject 015 

2 

1 

6 

5 

4 

6 

4 

3 

3 

2 

3 

4 

4 

3 


3.571429 

9 


Average 

2.43 

1.79 

6.29 

4.86 

3.79 

6.64 

3.50 

3.36 

2.71 

1.79 

2.29 

3.71 

4.71 

2.57 

4.14 




Rank 

4 

1 

14 

13 

10 

15 

8 

7 

6 

1 

3 

9 

12 

5 

11 






LOCATION ESTIMATION EVALUATIONS 



Subject Number 

001 

002 

003 

004 

005 

006 

007 

008 

009 

010 

oil 

012 

013 

014 

015 

Average 

Grade 

Harshness 

Ranking 

EVALUATORS I 

Subject 001 



5 

5 

4 

7 

4 

4 

3 

2 

3 

3 

4 

2 

3 

3.642857 

5 

Subject 002 

2 


7 

3 

7 

4 

4 

3 

2 

3 

2 

3 

2 

3 

3714286 

4 

Subject 003 

3 

3 

5 

4 

6 

4 

4 

3 

2 

3 

3 

4 

3 

3 

3,571429 

7 

Subject 004 

1 

4 

4 


3 

6 

2 

4 

1 

1 

1 

2 

3 

2 

2 

2571429 

15 

Subject 005 

2 

2 

5 

4 


7 

2 

3 

2 

2 

2 

2 

4 

1 

4 

3 

12 

Subject 006 

1 

2 

6 

5 

4 


3 

2 

2 

3 

4 

4 

4 

2 

3 

3.214286 

10 

Subject 007 

2 

2 

6 

5 

4 

7 


6 

3 

3 

3 

4 

5 

2 

5 

4.071429 

2 

Subject 008 

1 

1 

6 

4 

2 

7 

2 


2 

1 

2 

2 

4 

1 

3 

2.714286 

14 

Subject 009 

3 

2 

6 

5 

4 

6 

5 

5 


4 

3 

4 

4 

2 

5 

4.142857 

1 

Subject 010 

2 

1 

5 

4 

3 

6 

3 

3 

3 


4 

4 

3 

2 

4 

3.357143 

9 

Subject Oil 

2 

2 

5 

4 

5 

6 

4 

3 

3 

2 


3 

4 

2 

5 

3571429 

7 

Suited 012 

1 

1 

5 

6 

2 

6 

3 

3 

2 

1 

2 


4 

2 

2 

2.857143 

13 


Subject 013 

2 

2 

6 

5 

4 

6 

4 

3 

3 

3 

3 

4 


3 

3 

3642857 

5 


Subject 014 

2 

3 

7 

6 

4 

7 

4 

4 

2 

3 

3 

3 

4 

4 

4 

3 

Subject 015 

2 

2 

4 

4 

3 

6 

4 

3 

3 

2 

3 

3 

3 

3 3.214286 

10 


Average 

1.86 

2.07 

5.50 

493 

3.50 

6.43 

3.43 

364 

2.50 

2.21 

2.79 

3.07 

3.79 

2.07 

3.50 


Rank 

1 

2 

14 

13 

9 

15 

8 

11 

5 

4 

6 

7 

12 

2 

9 




OVERALL PERFORMANCE EVALUATIONS 



Subject Number 

001 

002 

003 

004 

006 

006 

007 

008 

009 

010 

Oil 

012 

013 

014 

015 

Average 

Grade 

Harshness 

Ranking 

EVALUATORS I 

Subject 001 


1 

0 

0 

1 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0.785714 

7 

Subject 002 


0 

0 

1 

0 



1 

1 



0 

1 

1 

0.714286 

9 

Subject 003 

1 

1 

0 

1 

0 



1 

1 




1 

1 

0.857143 

1 

Subject 004 

1 

1 

0 


1 

0 


1 

1 

1 


1 

1 

1 

1 

0.857143 

1 

Subject 005 

1 

1 

0 

1 


0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0857143 

1 

Subject 006 

1 

1 

0 

0 

1 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0857143 

1 

Subject 007 

1 

1 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0.857143 

1 

Subject 008 

1 

1 

0 


1 

0 

1 


1 

1 



0 

1 

1 

0.785714 

7 

Sutqect 009 

1 

1 

0 

0 

1 

0 

1 

1 


1 

1 

1 

0 

1 

0 

0.642857 

13 

Subject 010 

1 

1 

0 


1 

0 



1 




0 

1 

0 

0.714286 

9 

Subject Oil 

1 

1 

0 

0 

0 

0 



1 

1 



0 

1 

0 

0571429 

14 

Subject 012 

1 

1 

0 

0 

1 

0 

1 

1 

1 

1 

1 


0 

1 

1 

0714286 

9 

Subject 013 

1 

1 

0 

0 

0 

0 

0 

1 

1 

1 


1 


1 

0 

0,571429 

14 

Subject 014 

1 

1 

0 

0 

1 

0 

1 

1 

1 

1 

1 

1 

0 

1 

0.714286 

9 

Subject 015 

1 

1 

0 

1 

1 

0 

1 

1 

1 

1 

1 

1 

1 

1 


0857143 

1 


Sum 

14.00 

14.00 

0,00 

5.00 

12.00 

0.00 

13.00 

14.00 

14.00 

14.00 

14.00 

14.00 

7.00 

14.00 

10.00 


Average 

1.00 

1.00 

0,00 

036 

086 

000 

093 

1.00 

1.00 

100 

1.00 

1.00 

050 

1 00 

071 

Rank 

1 

1 

14 

13 

10 

14 

9 

1 

1 

1 

1 

1 

12 

1 

11 


Table 20. Subject pool's peer evaluations. 
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